2023-07-06T19:47:22.846 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-06T19:47:22.847 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} --archive /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147 --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/orig.config.yaml 2023-07-06T19:47:22.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T19:47:22.990 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147 branch: wip-rishabh-2023Jul5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} email: null first_in_suite: false job_id: '7328147' kernel: client: branch: testing flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-rishabh-2023Jul5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul5 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvVqYBqk6RwHd+gGqqcWtRbd74UJHsWYb4z5uVCfrZeMRhCv2tuwoMsQ044WHvlZMgS/Qazj6ja8+mE7864w35fl/CNZkDN3P4U7bwP8AAD5ZLAfagzgMFWLN+x0CLjKkyZB01iBc/aLgTvrtdvqaAHX9ifM1TVEo6/HsLd6lq/LUHz7VsJrZljDpyCDihvwM6FbCSlOS8UyiaNuJRAWX1YY+8QSmoX8EzVIgRh+6ZX2RlY7FIKC+WgsGT9/OvRIQ1T2Tjv0oOXfnyp2rghgplKgOX4Yk2mVWQF6QMHakH8r6DPwVRkgwOP33YYz5ljGabYsFdPkC8xgqtl1tetKgV1kPwM1iikRpfoTnyS53JiEyu5h1HNo+khB3DFNCqZcdl0eG8gsAJ2tXXWPCVa+Pz9F9xbfZzfM/OQsk0Grb6Xsv5y4Md/AN/qaK/JDSpho2W9QtQ296ujy37u7ufg/gagH2amLfwuHcvgQXIgs6G+sB2/CEDbTzX3qf4NKad3zs= smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbolPH+s+kUvv9GKmqZ24iklGKuOHmgaxXHyTohmTDlgsm5f43LwCq1OPDjKu6ZcN9ZRN24+VCCBQGETHv2hQFBvv2ut98zW19sVLuzpj1t8CFfr+1i2ZrRuU/NdFNuntkay/Z47cfg6ad7qIVH/EV9YA0G6XDCfHVN58isouGsLxWP3EmLDZnwxyLgGMUYn9I11qtLHCTu5Ru5lPUcbYuFm7I1NLiGSJU85I/lIjLEURU1awEB11ANyeBFIeZcr/+UawngE76MtYinp5PfiBKD/RXagAVXNioeD2CHpqqmKU/SfXK+ufDMby/PjnWL+lp+1wZ9b1Oqv0IXYJ5YTIk/kDdV9OPtK0CAGP3Dnn3Wf6optYpEsqpHsMbPaEF1xotK6ZVslPxNHpcPKYaRpWOiTOydPnvgIbttvBjKxzG21P6LSt/5YtikruWUfCUAzz8xbMNb7nWsACYRbkJAcmqkShJvYMuIMqI+PgV9tsN0fjSMR65yeYadmWpb+1LVTs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_quota teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-06_19:33:28 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-06T19:47:22.990 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa; will attempt to use it 2023-07-06T19:47:22.991 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks 2023-07-06T19:47:22.991 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-06T19:47:22.993 INFO:teuthology.task.internal:Checking packages... 2023-07-06T19:47:23.024 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '79365bb3b803780276d69efb2cc4afe5b160de9e' 2023-07-06T19:47:23.025 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-06T19:47:23.025 INFO:teuthology.packaging:ref: None 2023-07-06T19:47:23.026 INFO:teuthology.packaging:tag: None 2023-07-06T19:47:23.026 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul5 2023-07-06T19:47:23.026 INFO:teuthology.packaging:sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T19:47:23.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul5 2023-07-06T19:47:23.167 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4742.g79365bb3 2023-07-06T19:47:23.168 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-06T19:47:23.175 INFO:teuthology.task.internal:no buildpackages task found 2023-07-06T19:47:23.175 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-06T19:47:23.194 INFO:teuthology.task.internal:Saving configuration 2023-07-06T19:47:23.226 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-06T19:47:23.233 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T19:47:23.263 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:36.815553', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCfsqv+cTM+/p655yawGXNgktLwPUJsprC4d69ktcfShfaJ1bwpTLwxoA+lle83Z1hH5NpgmC9/WlE9LA7pMHyg='} 2023-07-06T19:47:23.290 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:36.817172', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0Nj7jKj4T7YA9e7NwE6r7RfSphEOxrs++T6mNaShYXFnNrg5eMDY4BZwVJsgjplUfhqQRDfqx7Gj7DCZZBJKg='} 2023-07-06T19:47:23.290 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-06T19:47:23.302 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-06T19:47:23.303 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-06T19:47:23.303 INFO:teuthology.run_tasks:Running task console_log... 2023-07-06T19:47:23.387 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4012d78280>, signals=[15]) 2023-07-06T19:47:23.389 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-06T19:47:23.396 INFO:teuthology.task.internal:Opening connections... 2023-07-06T19:47:23.397 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T19:47:23.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:47:23.472 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T19:47:23.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:47:23.539 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-06T19:47:23.545 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2023-07-06T19:47:23.568 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2023-07-06T19:47:23.568 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-07-06T19:47:23.630 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2023-07-06T19:47:23.630 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8" 2023-07-06T19:47:23.630 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2023-07-06T19:47:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2023-07-06T19:47:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8" 2023-07-06T19:47:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2023-07-06T19:47:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-06T19:47:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-07-06T19:47:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-06T19:47:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2023-07-06T19:47:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T19:47:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T19:47:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-06T19:47:23.633 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-07-06T19:47:23.655 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2023-07-06T19:47:23.678 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2023-07-06T19:47:23.678 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2023-07-06T19:47:23.739 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2023-07-06T19:47:23.739 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8" 2023-07-06T19:47:23.739 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2023-07-06T19:47:23.739 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2023-07-06T19:47:23.739 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T19:47:23.740 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T19:47:23.741 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-06T19:47:23.741 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2023-07-06T19:47:23.763 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-06T19:47:23.773 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-06T19:47:23.779 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-06T19:47:23.779 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2023-07-06T19:47:23.782 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2023-07-06T19:47:23.802 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-06T19:47:23.808 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-06T19:47:23.809 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2023-07-06T19:47:23.843 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2023-07-06T19:47:23.876 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-06T19:47:23.940 INFO:teuthology.run_tasks:Running task kernel... 2023-07-06T19:47:23.955 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-06T19:47:23.956 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-07-06T19:47:23.957 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-07-06T19:47:23.957 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T19:47:23.958 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T19:47:23.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:47:23.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-06T19:47:23.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:47:23.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-06T19:47:24.193 DEBUG:teuthology.task.kernel:sha1 for client.0 is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:47:24.193 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T19:47:24.194 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-07-06T19:47:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-489.el8.x86_64 2023-07-06T19:47:24.216 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:47:24.216 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T19:47:24.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2023-07-06T19:47:24.293 DEBUG:teuthology.task.kernel:sha1 for client.1 is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:47:24.294 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T19:47:24.294 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2023-07-06T19:47:24.315 INFO:teuthology.orchestra.run.smithi153.stdout:4.18.0-489.el8.x86_64 2023-07-06T19:47:24.316 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:47:24.316 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T19:47:24.316 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum upgrade -y linux-firmware 2023-07-06T19:47:28.111 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - AppStream 22 MB/s | 31 MB 00:01 2023-07-06T19:47:28.504 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - AppStream 20 MB/s | 31 MB 00:01 2023-07-06T19:47:35.153 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - BaseOS 28 MB/s | 41 MB 00:01 2023-07-06T19:47:35.895 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - BaseOS 27 MB/s | 41 MB 00:01 2023-07-06T19:47:41.294 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - Extras 73 kB/s | 18 kB 00:00 2023-07-06T19:47:41.920 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - Extras common packages 14 kB/s | 6.6 kB 00:00 2023-07-06T19:47:42.604 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras 45 kB/s | 18 kB 00:00 2023-07-06T19:47:43.422 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras common packages 10 kB/s | 6.6 kB 00:00 2023-07-06T19:47:44.171 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 1.6 kB/s | 3.5 kB 00:02 2023-07-06T19:47:46.090 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-06T19:47:46.599 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 16 MB 00:02 2023-07-06T19:47:48.273 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2023-07-06T19:47:50.405 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 447 kB/s | 24 kB 00:00 2023-07-06T19:47:52.207 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 618 kB/s | 24 kB 00:00 2023-07-06T19:47:53.860 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-07-06T19:47:53.860 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2023-07-06T19:47:53.861 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repo Size 2023-07-06T19:47:53.861 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2023-07-06T19:47:53.861 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2023-07-06T19:47:53.861 INFO:teuthology.orchestra.run.smithi153.stdout: linux-firmware noarch 20230515-115.gitd1962891.el8 baseos 266 M 2023-07-06T19:47:53.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T19:47:53.861 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-07-06T19:47:53.862 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2023-07-06T19:47:53.862 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 1 Package 2023-07-06T19:47:53.862 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T19:47:53.862 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 266 M 2023-07-06T19:47:53.862 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2023-07-06T19:47:55.835 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-07-06T19:47:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-07-06T19:47:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2023-07-06T19:47:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-07-06T19:47:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-07-06T19:47:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware noarch 20230515-115.gitd1962891.el8 baseos 266 M 2023-07-06T19:47:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T19:47:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-07-06T19:47:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-07-06T19:47:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 1 Package 2023-07-06T19:47:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T19:47:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 266 M 2023-07-06T19:47:55.838 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-07-06T19:48:03.572 INFO:teuthology.orchestra.run.smithi153.stdout:linux-firmware-20230515-115.gitd1962891.el8.noa 28 MB/s | 266 MB 00:09 2023-07-06T19:48:03.572 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2023-07-06T19:48:03.573 INFO:teuthology.orchestra.run.smithi153.stdout:Total 27 MB/s | 266 MB 00:09 2023-07-06T19:48:05.153 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2023-07-06T19:48:05.224 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2023-07-06T19:48:05.224 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2023-07-06T19:48:05.733 INFO:teuthology.orchestra.run.smithi027.stdout:linux-firmware-20230515-115.gitd1962891.el8.noa 28 MB/s | 266 MB 00:09 2023-07-06T19:48:05.733 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-07-06T19:48:05.734 INFO:teuthology.orchestra.run.smithi027.stdout:Total 27 MB/s | 266 MB 00:09 2023-07-06T19:48:06.880 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2023-07-06T19:48:06.903 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2023-07-06T19:48:07.388 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-07-06T19:48:07.449 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-07-06T19:48:07.450 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-07-06T19:48:09.181 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-07-06T19:48:09.219 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-07-06T19:48:24.617 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2023-07-06T19:48:24.988 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T19:48:25.240 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T19:48:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T19:48:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T19:48:26.204 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T19:48:26.204 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T19:48:26.205 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2023-07-06T19:48:26.205 INFO:teuthology.orchestra.run.smithi153.stdout: linux-firmware-20230515-115.gitd1962891.el8.noarch 2023-07-06T19:48:26.205 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T19:48:26.205 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-07-06T19:48:26.422 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on client.1... 2023-07-06T19:48:26.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:48:26.571 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T19:48:26.571 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T19:48:26.574 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi153.front.sepia.ceph.com to complete... 2023-07-06T19:48:28.438 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-07-06T19:48:28.803 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T19:48:29.069 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T19:48:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T19:48:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T19:48:30.064 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T19:48:30.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T19:48:30.064 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-07-06T19:48:30.065 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware-20230515-115.gitd1962891.el8.noarch 2023-07-06T19:48:30.065 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T19:48:30.065 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-07-06T19:48:30.271 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on client.0... 2023-07-06T19:48:30.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:48:30.486 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T19:48:30.486 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T19:48:30.489 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2023-07-06T19:48:44.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06 19:48:44 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T19:48:44.112 INFO:teuthology.orchestra.run.smithi153.stderr:FINISHED --2023-07-06 19:48:44-- 2023-07-06T19:48:44.112 INFO:teuthology.orchestra.run.smithi153.stderr:Total wall clock time: 18s 2023-07-06T19:48:44.113 INFO:teuthology.orchestra.run.smithi153.stderr:Downloaded: 1 files, 362M in 17s (21.0 MB/s) 2023-07-06T19:48:44.114 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on client.1... 2023-07-06T19:48:44.115 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T19:48:45.780 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying... ######################################## 2023-07-06T19:48:45.824 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing... ######################################## 2023-07-06T19:48:45.933 INFO:teuthology.orchestra.run.smithi153.stdout:Updating / installing... 2023-07-06T19:49:23.697 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T19:49:49.231 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T19:49:49.231 DEBUG:teuthology.orchestra.run.smithi153:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T19:49:49.285 INFO:teuthology.orchestra.run.smithi153.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T19:49:49.285 INFO:teuthology.orchestra.run.smithi153.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T19:49:49.286 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T19:49:49.287 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T19:49:49.288 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T19:49:49.289 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T19:49:49.290 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T19:49:49.291 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T19:49:49.292 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T19:49:49.293 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T19:49:49.294 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T19:49:49.295 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T19:49:49.296 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T19:49:49.297 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T19:49:49.298 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T19:49:49.299 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T19:49:49.300 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T19:49:49.301 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T19:49:49.302 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T19:49:49.303 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T19:49:49.304 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T19:49:49.305 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T19:49:49.306 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T19:49:49.307 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T19:49:49.308 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T19:49:49.309 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T19:49:49.310 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T19:49:49.311 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T19:49:49.312 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T19:49:49.313 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T19:49:49.314 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T19:49:49.315 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T19:49:49.316 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T19:49:49.317 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T19:49:49.318 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T19:49:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T19:49:49.320 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T19:49:49.321 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T19:49:49.322 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T19:49:49.323 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T19:49:49.324 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T19:49:49.325 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T19:49:49.326 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T19:49:49.327 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T19:49:49.328 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T19:49:49.329 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T19:49:49.330 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T19:49:49.331 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T19:49:49.332 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T19:49:49.333 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T19:49:49.334 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T19:49:49.335 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T19:49:49.336 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T19:49:49.337 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T19:49:49.338 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T19:49:49.339 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T19:49:49.340 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T19:49:49.341 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T19:49:49.342 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T19:49:49.343 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T19:49:49.344 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T19:49:49.345 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T19:49:49.346 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T19:49:49.347 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T19:49:49.348 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T19:49:49.349 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T19:49:49.350 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T19:49:49.351 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T19:49:49.352 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T19:49:49.353 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T19:49:49.354 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T19:49:49.355 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T19:49:49.356 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T19:49:49.357 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T19:49:49.358 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T19:49:49.359 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T19:49:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T19:49:49.361 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T19:49:49.362 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T19:49:49.363 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T19:49:49.364 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T19:49:49.365 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T19:49:49.366 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T19:49:49.367 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T19:49:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T19:49:49.369 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T19:49:49.370 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T19:49:49.371 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T19:49:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T19:49:49.373 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T19:49:49.374 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T19:49:49.375 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T19:49:49.376 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T19:49:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T19:49:49.378 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T19:49:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T19:49:49.380 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T19:49:49.381 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T19:49:49.382 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T19:49:49.383 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T19:49:49.384 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T19:49:49.385 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T19:49:49.386 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T19:49:49.387 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T19:49:49.388 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T19:49:49.389 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T19:49:49.390 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T19:49:49.391 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T19:49:49.392 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T19:49:49.393 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T19:49:49.394 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T19:49:49.395 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T19:49:49.396 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T19:49:49.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T19:49:49.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T19:49:49.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T19:49:49.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T19:49:49.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T19:49:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T19:49:49.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T19:49:49.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T19:49:49.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T19:49:49.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T19:49:49.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T19:49:49.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T19:49:49.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T19:49:49.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T19:49:49.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T19:49:49.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T19:49:49.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T19:49:49.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T19:49:49.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T19:49:49.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T19:49:49.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T19:49:49.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T19:49:49.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T19:49:49.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T19:49:49.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T19:49:49.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T19:49:49.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T19:49:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T19:49:49.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T19:49:49.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T19:49:49.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T19:49:49.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T19:49:49.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T19:49:49.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T19:49:49.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T19:49:49.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T19:49:49.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T19:49:49.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T19:49:49.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T19:49:49.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T19:49:49.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T19:49:49.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T19:49:49.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T19:49:49.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T19:49:49.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T19:49:49.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T19:49:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T19:49:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T19:49:49.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T19:49:49.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T19:49:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T19:49:49.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T19:49:49.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T19:49:49.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T19:49:49.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T19:49:49.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T19:49:49.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T19:49:49.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T19:49:49.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T19:49:49.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T19:49:49.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T19:49:49.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T19:49:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T19:49:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T19:49:49.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T19:49:49.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T19:49:49.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T19:49:49.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T19:49:49.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T19:49:49.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T19:49:49.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T19:49:49.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T19:49:49.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T19:49:49.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T19:49:49.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T19:49:49.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T19:49:49.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T19:49:49.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T19:49:49.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T19:49:49.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T19:49:49.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T19:49:49.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T19:49:49.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T19:49:49.480 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T19:49:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T19:49:49.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T19:49:49.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T19:49:49.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T19:49:49.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T19:49:49.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T19:49:49.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T19:49:49.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T19:49:49.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T19:49:49.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T19:49:49.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T19:49:49.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T19:49:49.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T19:49:49.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T19:49:49.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T19:49:49.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T19:49:49.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T19:49:49.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T19:49:49.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T19:49:49.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T19:49:49.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T19:49:49.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T19:49:49.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T19:49:49.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T19:49:49.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T19:49:49.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T19:49:49.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T19:49:49.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T19:49:49.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T19:49:49.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T19:49:49.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T19:49:49.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T19:49:49.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T19:49:49.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T19:49:49.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T19:49:49.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T19:49:49.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T19:49:49.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T19:49:49.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T19:49:49.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T19:49:49.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T19:49:49.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T19:49:49.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T19:49:49.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T19:49:49.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T19:49:49.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T19:49:49.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T19:49:49.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T19:49:49.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T19:49:49.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T19:49:49.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T19:49:49.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T19:49:49.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T19:49:49.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T19:49:49.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T19:49:49.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T19:49:49.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T19:49:49.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T19:49:49.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T19:49:49.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T19:49:49.536 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T19:49:49.536 DEBUG:teuthology.orchestra.run.smithi153:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T19:49:49.564 INFO:teuthology.orchestra.run.smithi153.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T19:49:49.564 INFO:teuthology.orchestra.run.smithi153.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:fi 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T19:49:49.565 INFO:teuthology.orchestra.run.smithi153.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:fi 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T19:49:49.566 INFO:teuthology.orchestra.run.smithi153.stdout:fi 2023-07-06T19:49:49.567 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -qi grub2-tools 2023-07-06T19:49:49.671 INFO:teuthology.orchestra.run.smithi153.stdout:Name : grub2-tools 2023-07-06T19:49:49.672 INFO:teuthology.orchestra.run.smithi153.stdout:Epoch : 1 2023-07-06T19:49:49.672 INFO:teuthology.orchestra.run.smithi153.stdout:Version : 2.02 2023-07-06T19:49:49.672 INFO:teuthology.orchestra.run.smithi153.stdout:Release : 129.el8 2023-07-06T19:49:49.673 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: x86_64 2023-07-06T19:49:49.673 INFO:teuthology.orchestra.run.smithi153.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-07-06T19:49:49.673 INFO:teuthology.orchestra.run.smithi153.stdout:Group : System Environment/Base 2023-07-06T19:49:49.673 INFO:teuthology.orchestra.run.smithi153.stdout:Size : 9265586 2023-07-06T19:49:49.674 INFO:teuthology.orchestra.run.smithi153.stdout:License : GPLv3+ 2023-07-06T19:49:49.674 INFO:teuthology.orchestra.run.smithi153.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-07-06T19:49:49.674 INFO:teuthology.orchestra.run.smithi153.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-07-06T19:49:49.675 INFO:teuthology.orchestra.run.smithi153.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-07-06T19:49:49.675 INFO:teuthology.orchestra.run.smithi153.stdout:Build Host : kbuilder.bsys.centos.org 2023-07-06T19:49:49.675 INFO:teuthology.orchestra.run.smithi153.stdout:Relocations : (not relocatable) 2023-07-06T19:49:49.675 INFO:teuthology.orchestra.run.smithi153.stdout:Packager : CentOS BuildSystem 2023-07-06T19:49:49.675 INFO:teuthology.orchestra.run.smithi153.stdout:Vendor : CentOS 2023-07-06T19:49:49.676 INFO:teuthology.orchestra.run.smithi153.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T19:49:49.676 INFO:teuthology.orchestra.run.smithi153.stdout:Summary : Support tools for GRUB. 2023-07-06T19:49:49.676 INFO:teuthology.orchestra.run.smithi153.stdout:Description : 2023-07-06T19:49:49.677 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T19:49:49.677 INFO:teuthology.orchestra.run.smithi153.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T19:49:49.677 INFO:teuthology.orchestra.run.smithi153.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T19:49:49.677 INFO:teuthology.orchestra.run.smithi153.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T19:49:49.678 INFO:teuthology.orchestra.run.smithi153.stdout:hardware devices. 2023-07-06T19:49:49.678 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T19:49:49.678 INFO:teuthology.orchestra.run.smithi153.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T19:49:49.679 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T19:49:49.680 INFO:teuthology.task.kernel:Updating grub on smithi153 to boot 6.4.0-g84139c18d56f 2023-07-06T19:49:49.680 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T19:49:52.250 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2023-07-06T19:49:54.464 INFO:teuthology.orchestra.run.smithi153.stderr:done 2023-07-06T19:49:54.466 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-07-06T19:49:54.502 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.P1vWduOpTd 2023-07-06T19:49:54.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.P1vWduOpTd 2023-07-06T19:49:54.576 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.P1vWduOpTd 2023-07-06T19:49:54.720 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.P1vWduOpTd is 6KB 2023-07-06T19:49:54.767 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.P1vWduOpTd 2023-07-06T19:49:54.784 DEBUG:teuthology.orchestra.run.smithi153:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T19:49:54.853 INFO:teuthology.orchestra.run.smithi153.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-07-06T19:49:54.854 INFO:teuthology.orchestra.run.smithi153.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-07-06T19:49:54.854 INFO:teuthology.orchestra.run.smithi153.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-07-06T19:49:54.854 INFO:teuthology.orchestra.run.smithi153.stdout:a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f.conf 2023-07-06T19:49:54.855 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f 2023-07-06T19:49:55.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2023-07-06T19:50:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06 19:50:08 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T19:50:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2023-07-06 19:50:08-- 2023-07-06T19:50:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 1m 38s 2023-07-06T19:50:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 362M in 1m 38s (3.71 MB/s) 2023-07-06T19:50:08.783 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on client.0... 2023-07-06T19:50:08.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T19:50:10.532 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2023-07-06T19:50:10.579 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2023-07-06T19:50:10.694 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2023-07-06T19:50:25.009 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-07-06T19:50:25.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T19:50:25.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:50:43.568 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2023-07-06T19:50:51.091 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T19:50:52.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T19:50:52.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:50:55.632 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2023-07-06T19:51:07.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T19:51:07.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:51:17.042 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T19:51:17.042 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T19:51:17.101 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T19:51:17.101 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T19:51:17.102 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T19:51:17.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T19:51:17.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T19:51:17.102 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T19:51:17.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T19:51:17.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T19:51:17.103 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T19:51:17.104 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T19:51:17.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T19:51:17.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T19:51:17.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T19:51:17.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T19:51:17.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T19:51:17.105 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T19:51:17.106 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T19:51:17.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T19:51:17.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T19:51:17.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T19:51:17.107 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T19:51:17.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T19:51:17.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T19:51:17.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T19:51:17.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T19:51:17.108 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T19:51:17.109 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T19:51:17.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T19:51:17.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T19:51:17.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T19:51:17.110 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T19:51:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T19:51:17.112 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T19:51:17.113 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T19:51:17.114 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T19:51:17.115 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T19:51:17.116 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T19:51:17.117 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T19:51:17.118 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T19:51:17.119 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T19:51:17.120 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T19:51:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T19:51:17.122 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T19:51:17.123 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T19:51:17.124 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T19:51:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T19:51:17.126 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T19:51:17.127 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T19:51:17.128 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T19:51:17.129 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T19:51:17.130 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T19:51:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T19:51:17.132 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T19:51:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T19:51:17.134 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T19:51:17.135 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T19:51:17.136 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T19:51:17.137 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T19:51:17.137 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T19:51:17.137 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T19:51:17.137 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T19:51:17.137 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T19:51:17.138 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T19:51:17.139 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T19:51:17.140 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T19:51:17.141 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T19:51:17.142 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T19:51:17.143 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T19:51:17.144 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T19:51:17.145 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T19:51:17.146 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T19:51:17.147 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T19:51:17.148 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T19:51:17.149 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T19:51:17.150 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T19:51:17.151 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T19:51:17.152 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T19:51:17.153 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T19:51:17.154 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T19:51:17.155 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T19:51:17.156 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T19:51:17.157 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T19:51:17.158 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T19:51:17.159 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T19:51:17.160 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T19:51:17.161 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T19:51:17.162 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T19:51:17.163 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T19:51:17.164 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T19:51:17.165 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T19:51:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T19:51:17.167 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T19:51:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T19:51:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T19:51:17.170 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T19:51:17.171 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T19:51:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T19:51:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T19:51:17.174 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T19:51:17.175 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T19:51:17.176 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T19:51:17.177 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T19:51:17.178 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T19:51:17.179 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T19:51:17.180 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T19:51:17.181 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T19:51:17.182 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T19:51:17.183 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T19:51:17.184 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T19:51:17.185 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T19:51:17.186 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T19:51:17.187 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T19:51:17.188 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T19:51:17.189 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T19:51:17.190 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T19:51:17.191 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T19:51:17.192 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T19:51:17.193 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T19:51:17.194 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T19:51:17.195 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T19:51:17.196 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T19:51:17.197 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T19:51:17.198 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T19:51:17.199 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T19:51:17.200 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T19:51:17.201 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T19:51:17.202 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T19:51:17.203 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T19:51:17.204 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T19:51:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T19:51:17.206 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T19:51:17.207 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T19:51:17.208 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T19:51:17.209 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T19:51:17.210 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T19:51:17.211 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T19:51:17.211 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T19:51:17.211 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T19:51:17.211 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T19:51:17.211 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T19:51:17.212 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T19:51:17.213 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T19:51:17.214 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T19:51:17.215 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T19:51:17.216 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T19:51:17.217 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T19:51:17.218 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T19:51:17.219 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T19:51:17.220 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T19:51:17.221 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T19:51:17.222 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T19:51:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T19:51:17.224 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T19:51:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T19:51:17.226 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T19:51:17.227 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T19:51:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T19:51:17.229 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T19:51:17.230 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T19:51:17.231 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T19:51:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T19:51:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T19:51:17.234 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T19:51:17.235 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T19:51:17.236 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T19:51:17.237 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T19:51:17.238 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T19:51:17.239 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T19:51:17.240 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T19:51:17.241 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T19:51:17.242 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T19:51:17.243 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T19:51:17.244 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T19:51:17.245 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T19:51:17.246 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T19:51:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T19:51:17.248 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T19:51:17.249 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T19:51:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T19:51:17.251 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T19:51:17.252 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T19:51:17.253 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T19:51:17.254 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T19:51:17.255 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T19:51:17.256 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T19:51:17.257 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T19:51:17.258 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T19:51:17.259 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T19:51:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T19:51:17.261 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T19:51:17.262 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T19:51:17.263 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T19:51:17.264 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T19:51:17.265 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T19:51:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T19:51:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T19:51:17.268 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T19:51:17.269 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T19:51:17.270 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T19:51:17.271 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T19:51:17.272 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T19:51:17.273 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T19:51:17.274 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T19:51:17.275 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T19:51:17.276 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T19:51:17.277 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T19:51:17.278 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T19:51:17.279 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T19:51:17.280 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T19:51:17.281 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T19:51:17.282 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T19:51:17.283 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T19:51:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T19:51:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T19:51:17.286 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T19:51:17.287 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T19:51:17.288 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T19:51:17.289 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T19:51:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T19:51:17.291 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T19:51:17.292 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T19:51:17.293 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T19:51:17.294 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T19:51:17.295 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T19:51:17.296 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T19:51:17.297 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T19:51:17.298 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T19:51:17.299 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T19:51:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T19:51:17.301 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T19:51:17.302 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T19:51:17.303 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T19:51:17.304 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T19:51:17.305 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T19:51:17.306 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T19:51:17.307 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T19:51:17.308 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T19:51:17.309 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T19:51:17.310 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T19:51:17.311 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T19:51:17.312 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T19:51:17.313 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T19:51:17.314 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T19:51:17.315 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T19:51:17.316 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T19:51:17.317 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T19:51:17.318 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T19:51:17.319 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T19:51:17.320 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T19:51:17.321 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T19:51:17.322 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T19:51:17.323 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T19:51:17.324 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T19:51:17.325 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T19:51:17.326 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T19:51:17.327 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T19:51:17.328 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T19:51:17.329 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T19:51:17.330 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T19:51:17.331 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T19:51:17.332 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T19:51:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T19:51:17.334 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T19:51:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T19:51:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T19:51:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T19:51:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T19:51:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T19:51:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T19:51:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T19:51:17.342 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T19:51:17.343 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T19:51:17.344 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T19:51:17.345 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T19:51:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T19:51:17.347 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T19:51:17.348 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T19:51:17.349 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T19:51:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T19:51:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T19:51:17.352 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T19:51:17.353 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T19:51:17.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T19:51:17.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T19:51:17.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T19:51:17.354 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T19:51:17.355 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T19:51:17.355 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T19:51:17.384 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:51:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-07-06T19:51:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T19:51:17.387 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T19:51:17.387 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T19:51:17.387 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-07-06T19:51:17.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2023-07-06T19:51:17.482 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2023-07-06T19:51:17.482 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2023-07-06T19:51:17.482 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.02 2023-07-06T19:51:17.482 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 129.el8 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:Group : System Environment/Base 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 9265586 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-07-06T19:51:17.483 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : kbuilder.bsys.centos.org 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:Relocations : (not relocatable) 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : CentOS BuildSystem 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T19:51:17.484 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2023-07-06T19:51:17.485 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2023-07-06T19:51:17.485 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T19:51:17.485 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T19:51:17.485 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T19:51:17.485 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T19:51:17.485 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2023-07-06T19:51:17.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T19:51:17.486 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T19:51:17.487 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T19:51:17.487 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.4.0-g84139c18d56f 2023-07-06T19:51:17.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T19:51:19.990 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2023-07-06T19:51:21.635 INFO:teuthology.orchestra.run.smithi027.stderr:done 2023-07-06T19:51:21.637 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-07-06T19:51:21.699 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.tJdpx67qoC 2023-07-06T19:51:21.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tJdpx67qoC 2023-07-06T19:51:21.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.tJdpx67qoC 2023-07-06T19:51:21.922 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.tJdpx67qoC is 6KB 2023-07-06T19:51:21.972 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.tJdpx67qoC 2023-07-06T19:51:21.990 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T19:51:22.081 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-07-06T19:51:22.081 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-07-06T19:51:22.081 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-07-06T19:51:22.081 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f.conf 2023-07-06T19:51:22.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f 2023-07-06T19:51:22.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2023-07-06T19:51:26.064 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2023-07-06T19:51:41.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T19:51:41.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:51:41.403 DEBUG:teuthology.orchestra.run.smithi153:> true 2023-07-06T19:51:41.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T19:51:41.825 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.4.0-g84139c18d56f"... 2023-07-06T19:51:41.826 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2023-07-06T19:51:41.872 INFO:teuthology.orchestra.run.smithi153.stdout:6.4.0-g84139c18d56f 2023-07-06T19:51:41.872 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T19:51:41.872 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T19:51:41.873 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-06T19:51:42.874 DEBUG:teuthology.parallel:result is None 2023-07-06T19:51:52.218 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-06T19:51:52.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T19:51:52.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:52:10.608 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-06T19:52:19.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T19:52:19.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:52:22.676 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-06T19:52:34.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T19:52:34.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:52:37.744 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-06T19:52:52.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T19:52:52.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:53:52.748 DEBUG:teuthology.orchestra.remote:timed out 2023-07-06T19:54:10.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T19:54:10.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:54:11.072 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-06T19:54:11.486 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T19:54:11.486 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.4.0-g84139c18d56f"... 2023-07-06T19:54:11.487 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-07-06T19:54:11.536 INFO:teuthology.orchestra.run.smithi027.stdout:6.4.0-g84139c18d56f 2023-07-06T19:54:11.536 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T19:54:11.536 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T19:54:11.538 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-06T19:54:12.540 DEBUG:teuthology.parallel:result is None 2023-07-06T19:54:12.541 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-06T19:54:12.575 INFO:teuthology.task.internal:Creating test directory... 2023-07-06T19:54:12.575 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T19:54:12.582 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T19:54:12.640 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-06T19:54:12.664 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-06T19:54:12.679 INFO:teuthology.task.internal:Creating archive directory... 2023-07-06T19:54:12.680 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T19:54:12.690 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T19:54:12.843 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-06T19:54:12.853 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-06T19:54:12.854 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T19:54:12.864 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T19:54:12.989 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:54:13.027 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:54:13.038 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:54:13.069 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:54:13.072 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-06T19:54:13.090 INFO:teuthology.task.internal:Configuring sudo... 2023-07-06T19:54:13.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T19:54:13.103 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T19:54:13.175 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-06T19:54:13.195 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-06T19:54:13.196 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T19:54:13.206 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T19:54:13.258 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:54:13.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:54:13.459 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:54:13.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:54:13.625 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T19:54:13.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T19:54:13.698 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:54:13.787 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:54:13.872 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:54:13.960 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:54:14.035 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T19:54:14.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T19:54:14.118 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2023-07-06T19:54:14.125 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2023-07-06T19:54:14.232 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T19:54:14.289 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T19:54:14.526 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-06T19:54:14.544 INFO:teuthology.task.internal:Starting timer... 2023-07-06T19:54:14.544 INFO:teuthology.run_tasks:Running task pcp... 2023-07-06T19:54:14.566 INFO:teuthology.run_tasks:Running task selinux... 2023-07-06T19:54:14.582 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-06T19:54:14.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2023-07-06T19:54:14.744 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2023-07-06T19:54:14.746 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2023-07-06T19:54:14.861 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2023-07-06T19:54:14.864 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-06T19:54:14.864 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2023-07-06T19:54:14.923 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2023-07-06T19:54:14.924 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2023-07-06T19:54:14.981 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2023-07-06T19:54:14.981 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2023-07-06T19:54:14.981 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T19:54:15.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:54:15.058 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T19:54:15.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:54:15.130 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-06T19:54:15.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2023-07-06T19:54:15.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2023-07-06T19:54:15.273 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-06T19:54:15.287 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-06T19:54:15.287 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-06T19:54:15.321 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-06T19:54:15.323 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-06T20:05:07.503 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2023-07-06T20:05:07.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T20:05:07.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T20:05:07.589 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-06T20:05:07.710 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-06T20:05:07.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T20:05:07.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T20:05:07.792 DEBUG:teuthology.orchestra.run.smithi153:> true 2023-07-06T20:05:07.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-07-06T20:05:07.910 INFO:teuthology.run_tasks:Running task clock... 2023-07-06T20:05:07.921 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-06T20:05:07.922 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T20:05:07.923 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:05:07.927 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T20:05:07.928 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:05:08.001 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T20:05:08.002 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T20:05:08.034 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T20:05:08.035 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T20:05:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2023-07-06T20:05:08.134 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2023-07-06T20:05:08.156 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2023-07-06T20:05:08.161 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2023-07-06T20:05:08.190 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T20:05:08.194 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T20:05:08.224 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T20:05:08.226 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T20:05:08.309 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-07-06T20:05:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2023-07-06T20:05:08.316 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:05:08.316 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-07-06T20:05:08.316 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.316 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.317 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.317 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.319 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:05:08.320 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-07-06T20:05:08.320 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.320 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.320 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.320 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:05:08.321 INFO:teuthology.run_tasks:Running task install... 2023-07-06T20:05:08.330 DEBUG:teuthology.task.install:project ceph 2023-07-06T20:05:08.331 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}} 2023-07-06T20:05:08.331 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'} 2023-07-06T20:05:08.332 INFO:teuthology.task.install:Using flavor: default 2023-07-06T20:05:08.341 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-06T20:05:08.341 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-06T20:05:08.342 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T20:05:08.343 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:05:08.345 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T20:05:08.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:05:08.482 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T20:05:08.483 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T20:05:08.491 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T20:05:08.491 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T20:05:08.597 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T20:05:08.597 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:05:08.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T20:05:08.670 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T20:05:08.670 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T20:05:08.709 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T20:05:08.710 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:05:08.710 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T20:05:08.780 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T20:05:08.781 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T20:05:08.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T20:05:08.881 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T20:05:08.889 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T20:05:08.951 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2023-07-06T20:05:08.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2023-07-06T20:05:08.985 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T20:05:09.052 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2023-07-06T20:05:09.055 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2023-07-06T20:05:09.322 INFO:teuthology.orchestra.run.smithi027.stdout:73 files removed 2023-07-06T20:05:09.358 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T20:05:09.399 INFO:teuthology.orchestra.run.smithi153.stdout:73 files removed 2023-07-06T20:05:09.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T20:05:10.336 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 206 kB/s | 81 kB 00:00 2023-07-06T20:05:10.425 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 180 kB/s | 81 kB 00:00 2023-07-06T20:05:10.603 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2023-07-06T20:05:10.684 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2023-07-06T20:05:11.101 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-07-06T20:05:11.204 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-07-06T20:05:13.062 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - AppStream 17 MB/s | 31 MB 00:01 2023-07-06T20:05:13.984 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - AppStream 11 MB/s | 31 MB 00:02 2023-07-06T20:05:21.446 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - BaseOS 20 MB/s | 41 MB 00:02 2023-07-06T20:05:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - BaseOS 20 MB/s | 41 MB 00:02 2023-07-06T20:05:28.455 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2023-07-06T20:05:28.860 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - Extras 35 kB/s | 18 kB 00:00 2023-07-06T20:05:29.298 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras common packages 10 kB/s | 6.6 kB 00:00 2023-07-06T20:05:29.561 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 6.6 kB 00:00 2023-07-06T20:05:30.657 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - PowerTools 5.2 MB/s | 6.3 MB 00:01 2023-07-06T20:05:30.839 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 8 - PowerTools 5.6 MB/s | 6.3 MB 00:01 2023-07-06T20:05:34.832 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T20:05:34.858 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T20:05:37.093 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 16 MB 00:02 2023-07-06T20:05:41.390 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 388 kB/s | 24 kB 00:00 2023-07-06T20:05:45.373 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:05:45.374 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:05:45.387 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-07-06T20:05:45.387 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-07-06T20:05:45.388 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-07-06T20:05:45.389 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-07-06T20:05:45.390 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T20:05:45.392 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T20:05:45.395 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T20:05:45.396 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T20:05:45.587 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-07-06T20:05:45.601 INFO:teuthology.orchestra.run.smithi153.stdout:==================================================================================================== 2023-07-06T20:05:45.601 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2023-07-06T20:05:45.601 INFO:teuthology.orchestra.run.smithi153.stdout:==================================================================================================== 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T20:05:45.602 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T20:05:45.603 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.189-2.el8 baseos 61 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T20:05:45.604 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.32.1-42.el8 baseos 99 k 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c appstream 1.6 M 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c appstream 903 k 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c appstream 5.1 M 2023-07-06T20:05:45.605 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-07-06T20:05:45.606 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-8.el8 baseos 143 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-debuginfod-client x86_64 0.189-2.el8 baseos 76 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf x86_64 0.189-2.el8 baseos 232 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libs x86_64 0.189-2.el8 baseos 302 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: libblkid x86_64 2.32.1-42.el8 baseos 220 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: libfdisk x86_64 2.32.1-42.el8 baseos 253 k 2023-07-06T20:05:45.607 INFO:teuthology.orchestra.run.smithi153.stdout: libmount x86_64 2.32.1-42.el8 baseos 236 k 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: libsmartcols x86_64 2.32.1-42.el8 baseos 179 k 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid x86_64 2.32.1-42.el8 baseos 98 k 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: util-linux x86_64 2.32.1-42.el8 baseos 2.6 M 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: util-linux-user x86_64 2.32.1-42.el8 baseos 102 k 2023-07-06T20:05:45.608 INFO:teuthology.orchestra.run.smithi153.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.1-7.el8 appstream 713 k 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T20:05:45.609 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-07-06T20:05:45.610 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-07-06T20:05:45.611 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-07-06T20:05:45.612 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-07-06T20:05:45.613 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T20:05:45.614 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-07-06T20:05:45.615 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T20:05:45.616 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-07-06T20:05:45.617 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T20:05:45.618 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T20:05:45.619 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-07-06T20:05:45.620 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling module streams: 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql 10 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout:==================================================================================================== 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout:Install 128 Packages 2023-07-06T20:05:45.621 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 16 Packages 2023-07-06T20:05:45.622 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:05:45.622 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 212 M 2023-07-06T20:05:45.622 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2023-07-06T20:05:46.583 INFO:teuthology.orchestra.run.smithi153.stdout:(1/144): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 42 kB/s | 6.4 kB 00:00 2023-07-06T20:05:46.825 INFO:teuthology.orchestra.run.smithi153.stdout:(2/144): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 3.4 MB/s | 847 kB 00:00 2023-07-06T20:05:46.925 INFO:teuthology.orchestra.run.smithi153.stdout:(3/144): ceph-immutable-object-cache-18.0.0-474 1.6 MB/s | 165 kB 00:00 2023-07-06T20:05:47.276 INFO:teuthology.orchestra.run.smithi153.stdout:(4/144): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 6.5 MB/s | 2.3 MB 00:00 2023-07-06T20:05:47.501 INFO:teuthology.orchestra.run.smithi153.stdout:(5/144): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 6.9 MB/s | 1.5 MB 00:00 2023-07-06T20:05:47.959 INFO:teuthology.orchestra.run.smithi153.stdout:(6/144): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 10 MB/s | 4.6 MB 00:00 2023-07-06T20:05:48.476 INFO:teuthology.orchestra.run.smithi153.stdout:(7/144): ceph-common-18.0.0-4742.g79365bb3.el8. 11 MB/s | 22 MB 00:02 2023-07-06T20:05:49.319 INFO:teuthology.orchestra.run.smithi153.stdout:(8/144): ceph-osd-18.0.0-4742.g79365bb3.el8.x86 13 MB/s | 18 MB 00:01 2023-07-06T20:05:49.368 INFO:teuthology.orchestra.run.smithi153.stdout:(9/144): ceph-selinux-18.0.0-4742.g79365bb3.el8 491 kB/s | 24 kB 00:00 2023-07-06T20:05:50.202 INFO:teuthology.orchestra.run.smithi153.stdout:(10/144): ceph-radosgw-18.0.0-4742.g79365bb3.el 7.8 MB/s | 13 MB 00:01 2023-07-06T20:05:50.603 INFO:teuthology.orchestra.run.smithi153.stdout:(11/144): cephfs-mirror-18.0.0-4742.g79365bb3.e 645 kB/s | 258 kB 00:00 2023-07-06T20:05:50.669 INFO:teuthology.orchestra.run.smithi153.stdout:(12/144): libcephfs-devel-18.0.0-4742.g79365bb3 467 kB/s | 31 kB 00:00 2023-07-06T20:05:50.953 INFO:teuthology.orchestra.run.smithi153.stdout:(13/144): ceph-base-18.0.0-4742.g79365bb3.el8.x 1.2 MB/s | 5.4 MB 00:04 2023-07-06T20:05:51.303 INFO:teuthology.orchestra.run.smithi153.stdout:(14/144): libcephfs2-18.0.0-4742.g79365bb3.el8. 1.2 MB/s | 770 kB 00:00 2023-07-06T20:05:51.329 INFO:teuthology.orchestra.run.smithi153.stdout:(15/144): libcephsqlite-18.0.0-4742.g79365bb3.e 366 kB/s | 137 kB 00:00 2023-07-06T20:05:51.512 INFO:teuthology.orchestra.run.smithi153.stdout:(16/144): librados-devel-18.0.0-4742.g79365bb3. 667 kB/s | 139 kB 00:00 2023-07-06T20:05:51.696 INFO:teuthology.orchestra.run.smithi153.stdout:(17/144): libradosstriper1-18.0.0-4742.g79365bb 1.3 MB/s | 475 kB 00:00 2023-07-06T20:05:51.788 INFO:teuthology.orchestra.run.smithi153.stdout:(18/144): python3-ceph-argparse-18.0.0-4742.g79 489 kB/s | 45 kB 00:00 2023-07-06T20:05:51.888 INFO:teuthology.orchestra.run.smithi153.stdout:(19/144): python3-ceph-common-18.0.0-4742.g7936 1.2 MB/s | 119 kB 00:00 2023-07-06T20:05:52.055 INFO:teuthology.orchestra.run.smithi153.stdout:(20/144): python3-cephfs-18.0.0-4742.g79365bb3. 1.2 MB/s | 197 kB 00:00 2023-07-06T20:05:52.238 INFO:teuthology.orchestra.run.smithi153.stdout:(21/144): python3-rados-18.0.0-4742.g79365bb3.e 2.0 MB/s | 370 kB 00:00 2023-07-06T20:05:52.422 INFO:teuthology.orchestra.run.smithi153.stdout:(22/144): python3-rbd-18.0.0-4742.g79365bb3.el8 1.9 MB/s | 364 kB 00:00 2023-07-06T20:05:52.522 INFO:teuthology.orchestra.run.smithi153.stdout:(23/144): python3-rgw-18.0.0-4742.g79365bb3.el8 1.0 MB/s | 107 kB 00:00 2023-07-06T20:05:52.631 INFO:teuthology.orchestra.run.smithi153.stdout:(24/144): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 870 kB/s | 94 kB 00:00 2023-07-06T20:05:53.365 INFO:teuthology.orchestra.run.smithi153.stdout:(25/144): ceph-test-18.0.0-4742.g79365bb3.el8.x 12 MB/s | 48 MB 00:03 2023-07-06T20:05:53.415 INFO:teuthology.orchestra.run.smithi153.stdout:(26/144): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 3.5 MB/s | 179 kB 00:00 2023-07-06T20:05:53.465 INFO:teuthology.orchestra.run.smithi153.stdout:(27/144): ceph-grafana-dashboards-18.0.0-4742.g 466 kB/s | 23 kB 00:00 2023-07-06T20:05:53.524 INFO:teuthology.orchestra.run.smithi153.stdout:(28/144): ceph-mgr-cephadm-18.0.0-4742.g79365bb 2.2 MB/s | 129 kB 00:00 2023-07-06T20:05:53.582 INFO:teuthology.orchestra.run.smithi153.stdout:(29/144): rbd-mirror-18.0.0-4742.g79365bb3.el8. 3.6 MB/s | 3.5 MB 00:00 2023-07-06T20:05:53.733 INFO:teuthology.orchestra.run.smithi153.stdout:(30/144): ceph-mgr-dashboard-18.0.0-4742.g79365 8.2 MB/s | 1.7 MB 00:00 2023-07-06T20:05:53.833 INFO:teuthology.orchestra.run.smithi153.stdout:(31/144): ceph-mgr-modules-core-18.0.0-4742.g79 2.4 MB/s | 240 kB 00:00 2023-07-06T20:05:53.900 INFO:teuthology.orchestra.run.smithi153.stdout:(32/144): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 713 kB/s | 47 kB 00:00 2023-07-06T20:05:53.959 INFO:teuthology.orchestra.run.smithi153.stdout:(33/144): ceph-prometheus-alerts-18.0.0-4742.g7 250 kB/s | 15 kB 00:00 2023-07-06T20:05:54.034 INFO:teuthology.orchestra.run.smithi153.stdout:(34/144): ceph-volume-18.0.0-4742.g79365bb3.el8 3.4 MB/s | 258 kB 00:00 2023-07-06T20:05:54.101 INFO:teuthology.orchestra.run.smithi153.stdout:(35/144): cephadm-18.0.0-4742.g79365bb3.el8.noa 2.5 MB/s | 166 kB 00:00 2023-07-06T20:05:54.167 INFO:teuthology.orchestra.run.smithi153.stdout:(36/144): cephfs-top-18.0.0-4742.g79365bb3.el8. 270 kB/s | 18 kB 00:00 2023-07-06T20:05:54.584 INFO:teuthology.orchestra.run.smithi153.stdout:(37/144): automake-1.16.1-7.el8.noarch.rpm 1.7 MB/s | 713 kB 00:00 2023-07-06T20:05:54.710 INFO:teuthology.orchestra.run.smithi153.stdout:(38/144): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2023-07-06T20:05:54.785 INFO:teuthology.orchestra.run.smithi153.stdout:(39/144): flex-2.6.1-9.el8.x86_64.rpm 4.2 MB/s | 320 kB 00:00 2023-07-06T20:05:54.861 INFO:teuthology.orchestra.run.smithi153.stdout:(40/144): jq-1.6-7.el8.x86_64.rpm 2.6 MB/s | 202 kB 00:00 2023-07-06T20:05:54.978 INFO:teuthology.orchestra.run.smithi153.stdout:(41/144): ceph-mgr-diskprediction-local-18.0.0- 5.3 MB/s | 7.4 MB 00:01 2023-07-06T20:05:55.020 INFO:teuthology.orchestra.run.smithi153.stdout:(42/144): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-07-06T20:05:55.112 INFO:teuthology.orchestra.run.smithi153.stdout:(43/144): libtool-2.4.6-25.el8.x86_64.rpm 7.6 MB/s | 709 kB 00:00 2023-07-06T20:05:55.178 INFO:teuthology.orchestra.run.smithi153.stdout:(44/144): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-07-06T20:05:55.237 INFO:teuthology.orchestra.run.smithi153.stdout:(45/144): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-07-06T20:05:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:(46/144): librdkafka-0.11.4-3.el8.x86_64.rpm 1.2 MB/s | 354 kB 00:00 2023-07-06T20:05:55.538 INFO:teuthology.orchestra.run.smithi153.stdout:(47/144): openblas-0.3.15-4.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-07-06T20:05:55.655 INFO:teuthology.orchestra.run.smithi153.stdout:(48/144): postgresql-10.23-1.module_el8+300+2c5 13 MB/s | 1.6 MB 00:00 2023-07-06T20:05:55.722 INFO:teuthology.orchestra.run.smithi153.stdout:(49/144): openblas-threads-0.3.15-4.el8.x86_64. 11 MB/s | 4.9 MB 00:00 2023-07-06T20:05:55.764 INFO:teuthology.orchestra.run.smithi153.stdout:(50/144): postgresql-contrib-10.23-1.module_el8 8.1 MB/s | 903 kB 00:00 2023-07-06T20:05:56.022 INFO:teuthology.orchestra.run.smithi153.stdout:(51/144): postgresql-server-10.23-1.module_el8+ 17 MB/s | 5.1 MB 00:00 2023-07-06T20:05:56.089 INFO:teuthology.orchestra.run.smithi153.stdout:(52/144): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-07-06T20:05:56.123 INFO:teuthology.orchestra.run.smithi153.stdout:(53/144): python3-jinja2-2.10.1-3.el8.noarch.rp 5.3 MB/s | 538 kB 00:00 2023-07-06T20:05:56.190 INFO:teuthology.orchestra.run.smithi153.stdout:(54/144): python3-jsonpatch-1.21-2.el8.noarch.r 265 kB/s | 27 kB 00:00 2023-07-06T20:05:56.215 INFO:teuthology.orchestra.run.smithi153.stdout:(55/144): python3-jsonpointer-1.10-11.el8.noarc 217 kB/s | 20 kB 00:00 2023-07-06T20:05:56.240 INFO:teuthology.orchestra.run.smithi153.stdout:(56/144): python3-mako-1.0.6-14.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-07-06T20:05:56.257 INFO:teuthology.orchestra.run.smithi153.stdout:(57/144): python3-markupsafe-0.23-19.el8.x86_64 930 kB/s | 39 kB 00:00 2023-07-06T20:05:56.324 INFO:teuthology.orchestra.run.smithi153.stdout:(58/144): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.3 MB/s | 225 kB 00:00 2023-07-06T20:05:56.399 INFO:teuthology.orchestra.run.smithi153.stdout:(59/144): python3-numpy-1.14.3-10.el8.x86_64.rp 24 MB/s | 3.7 MB 00:00 2023-07-06T20:05:56.416 INFO:teuthology.orchestra.run.smithi153.stdout:(60/144): python3-prettytable-0.7.2-14.el8.noar 478 kB/s | 44 kB 00:00 2023-07-06T20:05:56.450 INFO:teuthology.orchestra.run.smithi153.stdout:(61/144): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-06T20:05:56.475 INFO:teuthology.orchestra.run.smithi153.stdout:(62/144): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2023-07-06T20:05:56.501 INFO:teuthology.orchestra.run.smithi153.stdout:(63/144): python3-pyasn1-modules-0.3.7-6.el8.no 2.1 MB/s | 110 kB 00:00 2023-07-06T20:05:56.526 INFO:teuthology.orchestra.run.smithi153.stdout:(64/144): python3-pytz-2017.2-9.el8.noarch.rpm 1.0 MB/s | 54 kB 00:00 2023-07-06T20:05:56.601 INFO:teuthology.orchestra.run.smithi153.stdout:(65/144): python3-werkzeug-0.12.2-4.el8.noarch. 6.0 MB/s | 457 kB 00:00 2023-07-06T20:05:56.668 INFO:teuthology.orchestra.run.smithi153.stdout:(66/144): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-07-06T20:05:56.735 INFO:teuthology.orchestra.run.smithi153.stdout:(67/144): uuid-1.6.2-43.el8.x86_64.rpm 963 kB/s | 64 kB 00:00 2023-07-06T20:05:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:(68/144): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2023-07-06T20:05:56.944 INFO:teuthology.orchestra.run.smithi153.stdout:(69/144): python3-scipy-1.0.0-21.module_el8.5.0 31 MB/s | 14 MB 00:00 2023-07-06T20:05:57.111 INFO:teuthology.orchestra.run.smithi153.stdout:(70/144): elfutils-libelf-devel-0.189-2.el8.x86 194 kB/s | 61 kB 00:00 2023-07-06T20:05:57.136 INFO:teuthology.orchestra.run.smithi153.stdout:(71/144): fuse-2.9.7-17.el8.x86_64.rpm 433 kB/s | 83 kB 00:00 2023-07-06T20:05:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:(72/144): keyutils-libs-devel-1.5.10-9.el8.x86_ 576 kB/s | 48 kB 00:00 2023-07-06T20:05:57.262 INFO:teuthology.orchestra.run.smithi153.stdout:(73/144): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2023-07-06T20:05:57.321 INFO:teuthology.orchestra.run.smithi153.stdout:(74/144): krb5-devel-1.18.2-22.el8.x86_64.rpm 3.0 MB/s | 561 kB 00:00 2023-07-06T20:05:57.346 INFO:teuthology.orchestra.run.smithi153.stdout:(75/144): libcom_err-devel-1.45.6-5.el8.x86_64. 464 kB/s | 39 kB 00:00 2023-07-06T20:05:57.472 INFO:teuthology.orchestra.run.smithi153.stdout:(76/144): libgfortran-8.5.0-20.el8.x86_64.rpm 4.2 MB/s | 645 kB 00:00 2023-07-06T20:05:57.538 INFO:teuthology.orchestra.run.smithi153.stdout:(77/144): libquadmath-8.5.0-20.el8.x86_64.rpm 898 kB/s | 172 kB 00:00 2023-07-06T20:05:57.564 INFO:teuthology.orchestra.run.smithi153.stdout:(78/144): librabbitmq-0.9.0-4.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-07-06T20:05:57.630 INFO:teuthology.orchestra.run.smithi153.stdout:(79/144): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-07-06T20:05:57.656 INFO:teuthology.orchestra.run.smithi153.stdout:(80/144): libselinux-devel-2.9-8.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2023-07-06T20:05:57.731 INFO:teuthology.orchestra.run.smithi153.stdout:(81/144): librgw2-18.0.0-4742.g79365bb3.el8.x86 1.1 MB/s | 6.7 MB 00:06 2023-07-06T20:05:57.757 INFO:teuthology.orchestra.run.smithi153.stdout:(82/144): libverto-devel-0.3.2-2.el8.x86_64.rpm 180 kB/s | 18 kB 00:00 2023-07-06T20:05:57.782 INFO:teuthology.orchestra.run.smithi153.stdout:(83/144): libuuid-devel-2.32.1-42.el8.x86_64.rp 659 kB/s | 99 kB 00:00 2023-07-06T20:05:57.873 INFO:teuthology.orchestra.run.smithi153.stdout:(84/144): pcre2-devel-10.32-3.el8.x86_64.rpm 6.5 MB/s | 605 kB 00:00 2023-07-06T20:05:57.932 INFO:teuthology.orchestra.run.smithi153.stdout:(85/144): libzstd-devel-1.4.4-1.el8.x86_64.rpm 218 kB/s | 44 kB 00:00 2023-07-06T20:05:57.958 INFO:teuthology.orchestra.run.smithi153.stdout:(86/144): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-07-06T20:05:58.058 INFO:teuthology.orchestra.run.smithi153.stdout:(87/144): openssl-devel-1.1.1k-9.el8.x86_64.rpm 11 MB/s | 3.2 MB 00:00 2023-07-06T20:05:58.083 INFO:teuthology.orchestra.run.smithi153.stdout:(88/144): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 249 kB 00:00 2023-07-06T20:05:58.125 INFO:teuthology.orchestra.run.smithi153.stdout:(89/144): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-07-06T20:05:58.150 INFO:teuthology.orchestra.run.smithi153.stdout:(90/144): python3-jwt-1.6.1-2.el8.noarch.rpm 637 kB/s | 43 kB 00:00 2023-07-06T20:05:58.192 INFO:teuthology.orchestra.run.smithi153.stdout:(91/144): python3-cryptography-3.2.1-6.el8.x86_ 4.7 MB/s | 641 kB 00:00 2023-07-06T20:05:58.217 INFO:teuthology.orchestra.run.smithi153.stdout:(92/144): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-07-06T20:05:58.243 INFO:teuthology.orchestra.run.smithi153.stdout:(93/144): python3-oauthlib-2.1.0-1.el8.noarch.r 1.3 MB/s | 155 kB 00:00 2023-07-06T20:05:58.268 INFO:teuthology.orchestra.run.smithi153.stdout:(94/144): python3-requests-oauthlib-1.0.0-1.el8 564 kB/s | 43 kB 00:00 2023-07-06T20:05:58.294 INFO:teuthology.orchestra.run.smithi153.stdout:(95/144): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 982 kB/s | 74 kB 00:00 2023-07-06T20:05:58.319 INFO:teuthology.orchestra.run.smithi153.stdout:(96/144): zlib-devel-1.2.11-21.el8.x86_64.rpm 766 kB/s | 58 kB 00:00 2023-07-06T20:05:58.394 INFO:teuthology.orchestra.run.smithi153.stdout:(97/144): liboath-2.6.2-3.el8.x86_64.rpm 589 kB/s | 59 kB 00:00 2023-07-06T20:05:58.420 INFO:teuthology.orchestra.run.smithi153.stdout:(98/144): gperftools-libs-2.7-9.el8.x86_64.rpm 2.0 MB/s | 306 kB 00:00 2023-07-06T20:05:58.445 INFO:teuthology.orchestra.run.smithi153.stdout:(99/144): libunwind-1.3.1-3.el8.x86_64.rpm 598 kB/s | 75 kB 00:00 2023-07-06T20:05:58.470 INFO:teuthology.orchestra.run.smithi153.stdout:(100/144): python3-bcrypt-3.1.6-2.el8.1.x86_64. 873 kB/s | 44 kB 00:00 2023-07-06T20:05:58.512 INFO:teuthology.orchestra.run.smithi153.stdout:(101/144): python3-beautifulsoup4-4.6.3-2.el8.1 2.7 MB/s | 185 kB 00:00 2023-07-06T20:05:58.538 INFO:teuthology.orchestra.run.smithi153.stdout:(102/144): python3-cachetools-3.1.1-4.el8.noarc 492 kB/s | 33 kB 00:00 2023-07-06T20:05:58.563 INFO:teuthology.orchestra.run.smithi153.stdout:(103/144): python3-certifi-2018.10.15-7.el8.noa 325 kB/s | 16 kB 00:00 2023-07-06T20:05:58.588 INFO:teuthology.orchestra.run.smithi153.stdout:(104/144): python3-asyncssh-2.7.0-2.el8.noarch. 2.4 MB/s | 476 kB 00:00 2023-07-06T20:05:58.613 INFO:teuthology.orchestra.run.smithi153.stdout:(105/144): python3-cheroot-8.5.2-1.el8.noarch.r 2.2 MB/s | 173 kB 00:00 2023-07-06T20:05:58.638 INFO:teuthology.orchestra.run.smithi153.stdout:(106/144): python3-google-auth-1.1.1-10.el8.noa 1.6 MB/s | 81 kB 00:00 2023-07-06T20:05:58.655 INFO:teuthology.orchestra.run.smithi153.stdout:(107/144): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2023-07-06T20:05:58.672 INFO:teuthology.orchestra.run.smithi153.stdout:(108/144): python3-jaraco-functools-2.0-4.el8.n 527 kB/s | 18 kB 00:00 2023-07-06T20:05:58.706 INFO:teuthology.orchestra.run.smithi153.stdout:(109/144): python3-cherrypy-18.4.0-1.el8.noarch 2.6 MB/s | 384 kB 00:00 2023-07-06T20:05:58.731 INFO:teuthology.orchestra.run.smithi153.stdout:(110/144): python3-logutils-0.3.5-11.el8.noarch 831 kB/s | 49 kB 00:00 2023-07-06T20:05:58.756 INFO:teuthology.orchestra.run.smithi153.stdout:(111/144): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2023-07-06T20:05:58.782 INFO:teuthology.orchestra.run.smithi153.stdout:(112/144): python3-natsort-7.1.1-2.el8.noarch.r 1.2 MB/s | 60 kB 00:00 2023-07-06T20:05:58.807 INFO:teuthology.orchestra.run.smithi153.stdout:(113/144): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-07-06T20:05:58.832 INFO:teuthology.orchestra.run.smithi153.stdout:(114/144): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-07-06T20:05:58.857 INFO:teuthology.orchestra.run.smithi153.stdout:(115/144): python3-repoze-lru-0.7-6.el8.noarch. 671 kB/s | 34 kB 00:00 2023-07-06T20:05:58.883 INFO:teuthology.orchestra.run.smithi153.stdout:(116/144): python3-routes-2.4.1-12.el8.noarch.r 3.8 MB/s | 196 kB 00:00 2023-07-06T20:05:58.908 INFO:teuthology.orchestra.run.smithi153.stdout:(117/144): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-07-06T20:05:58.933 INFO:teuthology.orchestra.run.smithi153.stdout:(118/144): python3-simplegeneric-0.8.1-17.el8.n 373 kB/s | 19 kB 00:00 2023-07-06T20:05:58.959 INFO:teuthology.orchestra.run.smithi153.stdout:(119/144): python3-singledispatch-3.4.0.3-18.el 478 kB/s | 24 kB 00:00 2023-07-06T20:05:58.984 INFO:teuthology.orchestra.run.smithi153.stdout:(120/144): python3-tempora-1.14.1-5.el8.noarch. 586 kB/s | 29 kB 00:00 2023-07-06T20:05:59.009 INFO:teuthology.orchestra.run.smithi153.stdout:(121/144): python3-trustme-0.6.0-4.el8.noarch.r 549 kB/s | 27 kB 00:00 2023-07-06T20:05:59.043 INFO:teuthology.orchestra.run.smithi153.stdout:(122/144): python3-kubernetes-11.0.0-6.el8.noar 3.0 MB/s | 1.2 MB 00:00 2023-07-06T20:05:59.077 INFO:teuthology.orchestra.run.smithi153.stdout:(123/144): python3-waitress-1.4.3-1.el8.noarch. 2.7 MB/s | 255 kB 00:00 2023-07-06T20:05:59.102 INFO:teuthology.orchestra.run.smithi153.stdout:(124/144): python3-websocket-client-0.56.0-5.el 1.0 MB/s | 61 kB 00:00 2023-07-06T20:05:59.127 INFO:teuthology.orchestra.run.smithi153.stdout:(125/144): python3-webtest-2.0.33-1.el8.noarch. 1.7 MB/s | 85 kB 00:00 2023-07-06T20:05:59.152 INFO:teuthology.orchestra.run.smithi153.stdout:(126/144): python3-zc-lockfile-2.0-2.el8.noarch 459 kB/s | 23 kB 00:00 2023-07-06T20:05:59.178 INFO:teuthology.orchestra.run.smithi153.stdout:(127/144): python3-webob-1.8.5-1.el8.1.noarch.r 1.5 MB/s | 251 kB 00:00 2023-07-06T20:05:59.228 INFO:teuthology.orchestra.run.smithi153.stdout:(128/144): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-07-06T20:05:59.304 INFO:teuthology.orchestra.run.smithi153.stdout:(129/144): NetworkManager-initscripts-updown-1. 1.9 MB/s | 143 kB 00:00 2023-07-06T20:05:59.362 INFO:teuthology.orchestra.run.smithi153.stdout:(130/144): elfutils-debuginfod-client-0.189-2.e 1.3 MB/s | 76 kB 00:00 2023-07-06T20:05:59.429 INFO:teuthology.orchestra.run.smithi153.stdout:(131/144): elfutils-libelf-0.189-2.el8.x86_64.r 3.4 MB/s | 232 kB 00:00 2023-07-06T20:05:59.529 INFO:teuthology.orchestra.run.smithi153.stdout:(132/144): librados2-18.0.0-4742.g79365bb3.el8. 9.7 MB/s | 3.7 MB 00:00 2023-07-06T20:05:59.555 INFO:teuthology.orchestra.run.smithi153.stdout:(133/144): elfutils-libs-0.189-2.el8.x86_64.rpm 2.4 MB/s | 302 kB 00:00 2023-07-06T20:05:59.622 INFO:teuthology.orchestra.run.smithi153.stdout:(134/144): iproute-6.2.0-2.el8.x86_64.rpm 9.4 MB/s | 880 kB 00:00 2023-07-06T20:05:59.647 INFO:teuthology.orchestra.run.smithi153.stdout:(135/144): libblkid-2.32.1-42.el8.x86_64.rpm 2.4 MB/s | 220 kB 00:00 2023-07-06T20:05:59.689 INFO:teuthology.orchestra.run.smithi153.stdout:(136/144): libfdisk-2.32.1-42.el8.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2023-07-06T20:05:59.723 INFO:teuthology.orchestra.run.smithi153.stdout:(137/144): libmount-2.32.1-42.el8.x86_64.rpm 3.1 MB/s | 236 kB 00:00 2023-07-06T20:05:59.765 INFO:teuthology.orchestra.run.smithi153.stdout:(138/144): libsmartcols-2.32.1-42.el8.x86_64.rp 2.3 MB/s | 179 kB 00:00 2023-07-06T20:05:59.831 INFO:teuthology.orchestra.run.smithi153.stdout:(139/144): libuuid-2.32.1-42.el8.x86_64.rpm 908 kB/s | 98 kB 00:00 2023-07-06T20:05:59.932 INFO:teuthology.orchestra.run.smithi153.stdout:(140/144): util-linux-2.32.1-42.el8.x86_64.rpm 16 MB/s | 2.6 MB 00:00 2023-07-06T20:05:59.958 INFO:teuthology.orchestra.run.smithi153.stdout:(141/144): util-linux-user-2.32.1-42.el8.x86_64 816 kB/s | 102 kB 00:00 2023-07-06T20:06:00.016 INFO:teuthology.orchestra.run.smithi153.stdout:(142/144): xfsdump-3.1.8-6.el8.x86_64.rpm 3.9 MB/s | 334 kB 00:00 2023-07-06T20:06:00.091 INFO:teuthology.orchestra.run.smithi153.stdout:(143/144): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2023-07-06T20:06:01.974 INFO:teuthology.orchestra.run.smithi153.stdout:(144/144): librbd1-18.0.0-4742.g79365bb3.el8.x8 1.3 MB/s | 3.6 MB 00:02 2023-07-06T20:06:01.977 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:06:01.978 INFO:teuthology.orchestra.run.smithi153.stdout:Total 13 MB/s | 212 MB 00:16 2023-07-06T20:06:03.781 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2023-07-06T20:06:04.197 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2023-07-06T20:06:04.197 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2023-07-06T20:06:05.731 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2023-07-06T20:06:05.735 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2023-07-06T20:06:11.062 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2023-07-06T20:06:11.283 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-07-06T20:06:11.491 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/160 2023-07-06T20:06:11.860 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T20:06:11.968 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T20:06:12.135 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T20:06:12.225 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T20:06:12.415 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/160 2023-07-06T20:06:12.633 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : elfutils-libelf-0.189-2.el8.x86_64 5/160 2023-07-06T20:06:12.834 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/160 2023-07-06T20:06:12.993 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T20:06:13.445 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T20:06:13.643 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T20:06:14.071 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T20:06:14.268 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T20:06:14.415 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T20:06:14.618 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T20:06:14.734 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T20:06:14.928 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T20:06:15.068 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T20:06:15.295 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T20:06:15.505 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T20:06:15.598 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T20:06:15.827 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/160 2023-07-06T20:06:16.036 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/160 2023-07-06T20:06:16.485 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpq-13.5-1.el8.x86_64 16/160 2023-07-06T20:06:16.707 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/160 2023-07-06T20:06:16.851 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T20:06:16.978 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T20:06:17.134 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T20:06:17.278 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T20:06:17.488 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/160 2023-07-06T20:06:17.640 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/160 2023-07-06T20:06:17.751 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T20:06:17.906 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T20:06:18.043 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T20:06:18.719 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T20:06:18.901 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T20:06:19.580 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T20:06:19.809 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T20:06:20.651 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T20:06:20.995 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/160 2023-07-06T20:06:21.193 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/160 2023-07-06T20:06:21.364 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/160 2023-07-06T20:06:21.570 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/160 2023-07-06T20:06:22.125 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 30/160 2023-07-06T20:06:22.256 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 31/160 2023-07-06T20:06:22.496 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 32/160 2023-07-06T20:06:23.634 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/160 2023-07-06T20:06:24.003 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/160 2023-07-06T20:06:24.337 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/160 2023-07-06T20:06:24.533 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T20:06:26.681 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T20:06:27.022 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/160 2023-07-06T20:06:27.234 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/160 2023-07-06T20:06:27.446 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/160 2023-07-06T20:06:27.665 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 40/160 2023-07-06T20:06:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 41/160 2023-07-06T20:06:27.991 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : elfutils-libs-0.189-2.el8.x86_64 42/160 2023-07-06T20:06:28.151 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : elfutils-debuginfod-client-0.189-2.el8.x86_64 43/160 2023-07-06T20:06:28.374 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T20:06:28.457 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T20:06:28.649 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T20:06:28.733 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T20:06:28.879 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.32.1-42.el8.x86_64 46/160 2023-07-06T20:06:29.074 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T20:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T20:06:29.959 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T20:06:29.987 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T20:06:30.201 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/160 2023-07-06T20:06:30.475 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/160 2023-07-06T20:06:30.697 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/160 2023-07-06T20:06:30.871 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/160 2023-07-06T20:06:31.045 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/160 2023-07-06T20:06:31.255 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/160 2023-07-06T20:06:31.482 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/160 2023-07-06T20:06:31.730 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/160 2023-07-06T20:06:31.928 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/160 2023-07-06T20:06:32.089 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/160 2023-07-06T20:06:32.272 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/160 2023-07-06T20:06:32.465 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/160 2023-07-06T20:06:32.635 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/160 2023-07-06T20:06:32.812 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/160 2023-07-06T20:06:33.063 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/160 2023-07-06T20:06:33.332 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/160 2023-07-06T20:06:33.541 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/160 2023-07-06T20:06:33.892 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/160 2023-07-06T20:06:34.201 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/160 2023-07-06T20:06:34.404 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/160 2023-07-06T20:06:35.217 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/160 2023-07-06T20:06:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T20:06:35.526 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T20:06:35.760 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 71/160 2023-07-06T20:06:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 72/160 2023-07-06T20:06:36.446 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 73/160 2023-07-06T20:06:36.756 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 74/160 2023-07-06T20:06:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 75/160 2023-07-06T20:06:37.270 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 76/160 2023-07-06T20:06:37.517 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 77/160 2023-07-06T20:06:38.857 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/160 2023-07-06T20:06:39.264 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/160 2023-07-06T20:06:39.508 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 80/160 2023-07-06T20:06:39.769 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/160 2023-07-06T20:06:40.074 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/160 2023-07-06T20:06:40.309 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/160 2023-07-06T20:06:40.527 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/160 2023-07-06T20:06:40.732 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/160 2023-07-06T20:06:41.074 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/160 2023-07-06T20:06:41.295 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/160 2023-07-06T20:06:41.450 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/160 2023-07-06T20:06:41.642 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/160 2023-07-06T20:06:41.983 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/160 2023-07-06T20:06:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/160 2023-07-06T20:06:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/160 2023-07-06T20:06:42.624 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/160 2023-07-06T20:06:42.833 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/160 2023-07-06T20:06:43.059 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/160 2023-07-06T20:06:43.267 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/160 2023-07-06T20:06:43.496 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T20:06:43.620 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T20:06:43.807 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 98/160 2023-07-06T20:06:43.956 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 99/160 2023-07-06T20:06:46.800 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T20:06:47.089 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T20:06:47.987 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T20:06:48.273 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T20:06:48.473 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T20:06:48.679 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T20:07:09.270 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T20:07:09.270 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2023-07-06T20:07:09.270 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2023-07-06T20:07:09.270 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /run 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-07-06T20:07:09.271 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:07:09.668 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T20:07:11.408 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T20:07:11.652 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T20:07:11.918 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T20:07:12.131 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T20:07:12.661 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T20:07:12.884 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T20:07:12.933 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T20:07:13.143 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 107/160 2023-07-06T20:07:13.364 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 108/160 2023-07-06T20:07:13.526 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T20:07:13.623 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T20:07:13.938 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : jq-1.6-7.el8.x86_64 110/160 2023-07-06T20:07:14.167 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.1-7.el8.noarch 111/160 2023-07-06T20:07:14.785 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T20:07:14.980 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T20:07:15.116 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 113/160 2023-07-06T20:07:15.356 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 114/160 2023-07-06T20:07:15.608 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T20:07:15.869 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T20:07:16.101 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T20:07:17.216 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T20:07:17.408 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T20:07:17.539 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T20:07:17.997 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 118/160 2023-07-06T20:07:18.193 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T20:07:18.258 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T20:07:18.425 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T20:07:18.451 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T20:07:18.646 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 121/160 2023-07-06T20:07:18.799 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T20:07:23.733 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T20:07:25.254 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 123/160 2023-07-06T20:07:25.492 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T20:07:25.556 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T20:07:25.724 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T20:07:25.799 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T20:07:26.024 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T20:07:26.432 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T20:07:26.641 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T20:07:26.929 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T20:07:27.299 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 128/160 2023-07-06T20:07:28.552 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T20:07:28.862 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 130/160 2023-07-06T20:07:29.103 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.189-2.el8.x86_64 131/160 2023-07-06T20:07:29.312 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : util-linux-user-2.32.1-42.el8.x86_64 132/160 2023-07-06T20:07:29.511 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/160 2023-07-06T20:07:29.724 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 134/160 2023-07-06T20:07:30.839 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T20:07:31.098 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T20:07:31.187 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T20:07:31.428 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 136/160 2023-07-06T20:07:31.690 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 137/160 2023-07-06T20:07:31.890 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 138/160 2023-07-06T20:07:32.240 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 139/160 2023-07-06T20:07:32.491 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 140/160 2023-07-06T20:07:32.656 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T20:07:32.685 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T20:07:32.901 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 142/160 2023-07-06T20:07:33.147 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T20:07:33.260 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T20:07:33.522 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T20:07:33.548 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T20:07:33.757 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 145/160 2023-07-06T20:07:33.967 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 146/160 2023-07-06T20:07:34.151 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 147/160 2023-07-06T20:07:34.152 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T20:07:34.443 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T20:07:34.444 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T20:07:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T20:07:34.690 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T20:07:34.803 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T20:07:34.804 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T20:07:35.054 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T20:07:35.275 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/160 2023-07-06T20:07:35.442 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/160 2023-07-06T20:07:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T20:07:35.682 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T20:07:35.900 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T20:07:35.900 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T20:07:36.125 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T20:07:36.125 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T20:07:36.350 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T20:07:36.350 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T20:07:36.566 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T20:07:36.567 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T20:07:36.808 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T20:07:37.015 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T20:07:37.237 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T20:07:37.238 INFO:teuthology.orchestra.run.smithi153.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T20:07:37.238 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:07:38.003 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/160 2023-07-06T20:07:38.003 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/160 2023-07-06T20:07:38.003 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/160 2023-07-06T20:07:38.003 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/160 2023-07-06T20:07:38.004 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/160 2023-07-06T20:07:38.004 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/160 2023-07-06T20:07:38.004 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/160 2023-07-06T20:07:38.004 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/160 2023-07-06T20:07:38.005 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/160 2023-07-06T20:07:38.006 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/160 2023-07-06T20:07:38.006 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/160 2023-07-06T20:07:38.017 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/160 2023-07-06T20:07:38.018 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/160 2023-07-06T20:07:38.019 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/160 2023-07-06T20:07:38.020 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/160 2023-07-06T20:07:38.020 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/160 2023-07-06T20:07:38.020 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/160 2023-07-06T20:07:38.020 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/160 2023-07-06T20:07:38.020 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : jq-1.6-7.el8.x86_64 42/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/160 2023-07-06T20:07:38.021 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/160 2023-07-06T20:07:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/160 2023-07-06T20:07:38.023 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/160 2023-07-06T20:07:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/160 2023-07-06T20:07:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/160 2023-07-06T20:07:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/160 2023-07-06T20:07:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/160 2023-07-06T20:07:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/160 2023-07-06T20:07:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/160 2023-07-06T20:07:38.025 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/160 2023-07-06T20:07:38.025 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/160 2023-07-06T20:07:38.025 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/160 2023-07-06T20:07:38.025 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.189-2.el8.x86_64 71/160 2023-07-06T20:07:38.025 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 77/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 78/160 2023-07-06T20:07:38.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.32.1-42.el8.x86_64 82/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 85/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/160 2023-07-06T20:07:38.027 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/160 2023-07-06T20:07:38.028 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/160 2023-07-06T20:07:38.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/160 2023-07-06T20:07:38.030 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/160 2023-07-06T20:07:38.030 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/160 2023-07-06T20:07:38.030 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/160 2023-07-06T20:07:38.030 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/160 2023-07-06T20:07:38.030 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/160 2023-07-06T20:07:38.031 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/160 2023-07-06T20:07:38.031 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/160 2023-07-06T20:07:38.031 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/160 2023-07-06T20:07:38.031 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/160 2023-07-06T20:07:38.031 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/160 2023-07-06T20:07:38.031 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 118/160 2023-07-06T20:07:38.032 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/160 2023-07-06T20:07:38.033 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/160 2023-07-06T20:07:38.033 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/160 2023-07-06T20:07:38.033 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/160 2023-07-06T20:07:38.033 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/160 2023-07-06T20:07:38.034 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/160 2023-07-06T20:07:38.034 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/160 2023-07-06T20:07:38.034 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/160 2023-07-06T20:07:38.034 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/160 2023-07-06T20:07:38.035 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/160 2023-07-06T20:07:38.035 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T20:07:38.035 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/160 2023-07-06T20:07:38.036 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/160 2023-07-06T20:07:38.036 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/160 2023-07-06T20:07:38.036 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-8.el 133/160 2023-07-06T20:07:38.036 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/160 2023-07-06T20:07:38.037 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-debuginfod-client-0.189-2.el8.x86_64 135/160 2023-07-06T20:07:38.037 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 136/160 2023-07-06T20:07:38.037 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-0.189-2.el8.x86_64 137/160 2023-07-06T20:07:38.038 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 138/160 2023-07-06T20:07:38.038 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libs-0.189-2.el8.x86_64 139/160 2023-07-06T20:07:38.038 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 140/160 2023-07-06T20:07:38.038 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 141/160 2023-07-06T20:07:38.039 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 142/160 2023-07-06T20:07:38.039 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libblkid-2.32.1-42.el8.x86_64 143/160 2023-07-06T20:07:38.039 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 144/160 2023-07-06T20:07:38.039 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libfdisk-2.32.1-42.el8.x86_64 145/160 2023-07-06T20:07:38.040 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 146/160 2023-07-06T20:07:38.040 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libmount-2.32.1-42.el8.x86_64 147/160 2023-07-06T20:07:38.040 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 148/160 2023-07-06T20:07:38.041 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libsmartcols-2.32.1-42.el8.x86_64 149/160 2023-07-06T20:07:38.041 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 150/160 2023-07-06T20:07:38.041 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-2.32.1-42.el8.x86_64 151/160 2023-07-06T20:07:38.042 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 152/160 2023-07-06T20:07:38.042 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : util-linux-2.32.1-42.el8.x86_64 153/160 2023-07-06T20:07:38.042 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T20:07:38.042 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : util-linux-user-2.32.1-42.el8.x86_64 155/160 2023-07-06T20:07:38.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/160 2023-07-06T20:07:38.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 157/160 2023-07-06T20:07:38.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 158/160 2023-07-06T20:07:38.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 159/160 2023-07-06T20:07:40.131 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 160/160 2023-07-06T20:07:40.132 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:07:40.132 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2023-07-06T20:07:40.132 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-initscripts-updown-1:1.40.16-8.el8.noarch 2023-07-06T20:07:40.132 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-debuginfod-client-0.189-2.el8.x86_64 2023-07-06T20:07:40.132 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-0.189-2.el8.x86_64 2023-07-06T20:07:40.132 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libs-0.189-2.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: iproute-6.2.0-2.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: libblkid-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: libfdisk-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: libmount-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: libsmartcols-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.133 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout: util-linux-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout: util-linux-user-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T20:07:40.134 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.136 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.136 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.136 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.136 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.136 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.136 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.137 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.189-2.el8.x86_64 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.7-17.el8.x86_64 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: jq-1.6-7.el8.x86_64 2023-07-06T20:07:40.138 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-07-06T20:07:40.139 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T20:07:40.140 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.141 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-07-06T20:07:40.141 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T20:07:40.141 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T20:07:40.141 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T20:07:40.141 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.32.1-42.el8.x86_64 2023-07-06T20:07:40.141 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-07-06T20:07:40.142 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-07-06T20:07:40.142 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T20:07:40.142 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T20:07:40.142 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.15-4.el8.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T20:07:40.143 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T20:07:40.144 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T20:07:40.145 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-07-06T20:07:40.146 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T20:07:40.146 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T20:07:40.146 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T20:07:40.146 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T20:07:40.146 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T20:07:40.146 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.0.6-14.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T20:07:40.147 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T20:07:40.148 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T20:07:40.149 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.150 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T20:07:40.150 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T20:07:40.150 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-07-06T20:07:40.150 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T20:07:40.150 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T20:07:40.151 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T20:07:40.151 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T20:07:40.151 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T20:07:40.151 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T20:07:40.151 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:07:40.152 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T20:07:40.153 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T20:07:40.153 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T20:07:40.153 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-07-06T20:07:40.153 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T20:07:40.153 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-07-06T20:07:40.154 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:07:40.154 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-07-06T20:07:40.460 DEBUG:teuthology.parallel:result is None 2023-07-06T20:10:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 59 kB/s | 16 MB 04:32 2023-07-06T20:10:12.446 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 302 kB/s | 24 kB 00:00 2023-07-06T20:10:16.638 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:10:16.639 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:10:16.652 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-07-06T20:10:16.652 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-07-06T20:10:16.653 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-07-06T20:10:16.654 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-07-06T20:10:16.655 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T20:10:16.657 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T20:10:16.660 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T20:10:16.661 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T20:10:16.867 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-07-06T20:10:16.883 INFO:teuthology.orchestra.run.smithi027.stdout:==================================================================================================== 2023-07-06T20:10:16.883 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-07-06T20:10:16.883 INFO:teuthology.orchestra.run.smithi027.stdout:==================================================================================================== 2023-07-06T20:10:16.883 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T20:10:16.884 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T20:10:16.885 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.189-2.el8 baseos 61 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T20:10:16.886 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.32.1-42.el8 baseos 99 k 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c appstream 1.6 M 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c appstream 903 k 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c appstream 5.1 M 2023-07-06T20:10:16.887 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-07-06T20:10:16.888 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-8.el8 baseos 143 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client x86_64 0.189-2.el8 baseos 76 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.189-2.el8 baseos 232 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.189-2.el8 baseos 302 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid x86_64 2.32.1-42.el8 baseos 220 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk x86_64 2.32.1-42.el8 baseos 253 k 2023-07-06T20:10:16.889 INFO:teuthology.orchestra.run.smithi027.stdout: libmount x86_64 2.32.1-42.el8 baseos 236 k 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols x86_64 2.32.1-42.el8 baseos 179 k 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid x86_64 2.32.1-42.el8 baseos 98 k 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux x86_64 2.32.1-42.el8 baseos 2.6 M 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user x86_64 2.32.1-42.el8 baseos 102 k 2023-07-06T20:10:16.890 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.1-7.el8 appstream 713 k 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T20:10:16.891 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-07-06T20:10:16.892 INFO:teuthology.orchestra.run.smithi027.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-07-06T20:10:16.893 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-07-06T20:10:16.894 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-07-06T20:10:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T20:10:16.896 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T20:10:16.897 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-07-06T20:10:16.898 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-07-06T20:10:16.899 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T20:10:16.900 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T20:10:16.901 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-07-06T20:10:16.902 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling module streams: 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql 10 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout:==================================================================================================== 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout:Install 128 Packages 2023-07-06T20:10:16.903 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 16 Packages 2023-07-06T20:10:16.904 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:10:16.904 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 212 M 2023-07-06T20:10:16.904 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-07-06T20:10:17.408 INFO:teuthology.orchestra.run.smithi027.stdout:(1/144): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-07-06T20:10:18.524 INFO:teuthology.orchestra.run.smithi027.stdout:(2/144): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 759 kB/s | 847 kB 00:01 2023-07-06T20:10:18.758 INFO:teuthology.orchestra.run.smithi027.stdout:(3/144): ceph-immutable-object-cache-18.0.0-474 709 kB/s | 165 kB 00:00 2023-07-06T20:10:19.975 INFO:teuthology.orchestra.run.smithi027.stdout:(4/144): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-07-06T20:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout:(5/144): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 4.0 MB/s | 1.5 MB 00:00 2023-07-06T20:10:21.025 INFO:teuthology.orchestra.run.smithi027.stdout:(6/144): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 7.0 MB/s | 4.6 MB 00:00 2023-07-06T20:10:23.242 INFO:teuthology.orchestra.run.smithi027.stdout:(7/144): ceph-osd-18.0.0-4742.g79365bb3.el8.x86 8.0 MB/s | 18 MB 00:02 2023-07-06T20:10:23.500 INFO:teuthology.orchestra.run.smithi027.stdout:(8/144): ceph-base-18.0.0-4742.g79365bb3.el8.x8 873 kB/s | 5.4 MB 00:06 2023-07-06T20:10:23.626 INFO:teuthology.orchestra.run.smithi027.stdout:(9/144): ceph-selinux-18.0.0-4742.g79365bb3.el8 194 kB/s | 24 kB 00:00 2023-07-06T20:10:27.018 INFO:teuthology.orchestra.run.smithi027.stdout:(10/144): ceph-radosgw-18.0.0-4742.g79365bb3.el 3.6 MB/s | 13 MB 00:03 2023-07-06T20:10:27.467 INFO:teuthology.orchestra.run.smithi027.stdout:(11/144): cephfs-mirror-18.0.0-4742.g79365bb3.e 574 kB/s | 258 kB 00:00 2023-07-06T20:10:27.835 INFO:teuthology.orchestra.run.smithi027.stdout:(12/144): libcephfs-devel-18.0.0-4742.g79365bb3 84 kB/s | 31 kB 00:00 2023-07-06T20:10:29.043 INFO:teuthology.orchestra.run.smithi027.stdout:(13/144): libcephfs2-18.0.0-4742.g79365bb3.el8. 638 kB/s | 770 kB 00:01 2023-07-06T20:10:29.618 INFO:teuthology.orchestra.run.smithi027.stdout:(14/144): libcephsqlite-18.0.0-4742.g79365bb3.e 239 kB/s | 137 kB 00:00 2023-07-06T20:10:29.902 INFO:teuthology.orchestra.run.smithi027.stdout:(15/144): librados-devel-18.0.0-4742.g79365bb3. 491 kB/s | 139 kB 00:00 2023-07-06T20:10:30.760 INFO:teuthology.orchestra.run.smithi027.stdout:(16/144): libradosstriper1-18.0.0-4742.g79365bb 553 kB/s | 475 kB 00:00 2023-07-06T20:10:38.501 INFO:teuthology.orchestra.run.smithi027.stdout:(17/144): librgw2-18.0.0-4742.g79365bb3.el8.x86 891 kB/s | 6.7 MB 00:07 2023-07-06T20:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:(18/144): python3-ceph-argparse-18.0.0-4742.g79 674 kB/s | 45 kB 00:00 2023-07-06T20:10:38.669 INFO:teuthology.orchestra.run.smithi027.stdout:(19/144): python3-ceph-common-18.0.0-4742.g7936 1.2 MB/s | 119 kB 00:00 2023-07-06T20:10:38.852 INFO:teuthology.orchestra.run.smithi027.stdout:(20/144): python3-cephfs-18.0.0-4742.g79365bb3. 1.1 MB/s | 197 kB 00:00 2023-07-06T20:10:39.244 INFO:teuthology.orchestra.run.smithi027.stdout:(21/144): python3-rados-18.0.0-4742.g79365bb3.e 946 kB/s | 370 kB 00:00 2023-07-06T20:10:39.628 INFO:teuthology.orchestra.run.smithi027.stdout:(22/144): python3-rbd-18.0.0-4742.g79365bb3.el8 951 kB/s | 364 kB 00:00 2023-07-06T20:10:39.811 INFO:teuthology.orchestra.run.smithi027.stdout:(23/144): python3-rgw-18.0.0-4742.g79365bb3.el8 582 kB/s | 107 kB 00:00 2023-07-06T20:10:39.978 INFO:teuthology.orchestra.run.smithi027.stdout:(24/144): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 566 kB/s | 94 kB 00:00 2023-07-06T20:10:43.570 INFO:teuthology.orchestra.run.smithi027.stdout:(25/144): rbd-mirror-18.0.0-4742.g79365bb3.el8. 984 kB/s | 3.5 MB 00:03 2023-07-06T20:10:43.796 INFO:teuthology.orchestra.run.smithi027.stdout:(26/144): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 798 kB/s | 179 kB 00:00 2023-07-06T20:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout:(27/144): ceph-grafana-dashboards-18.0.0-4742.g 173 kB/s | 23 kB 00:00 2023-07-06T20:10:44.146 INFO:teuthology.orchestra.run.smithi027.stdout:(28/144): ceph-common-18.0.0-4742.g79365bb3.el8 838 kB/s | 22 MB 00:26 2023-07-06T20:10:44.188 INFO:teuthology.orchestra.run.smithi027.stdout:(29/144): ceph-mgr-cephadm-18.0.0-4742.g79365bb 498 kB/s | 129 kB 00:00 2023-07-06T20:10:45.964 INFO:teuthology.orchestra.run.smithi027.stdout:(30/144): ceph-mgr-dashboard-18.0.0-4742.g79365 964 kB/s | 1.7 MB 00:01 2023-07-06T20:10:46.464 INFO:teuthology.orchestra.run.smithi027.stdout:(31/144): ceph-mgr-modules-core-18.0.0-4742.g79 481 kB/s | 240 kB 00:00 2023-07-06T20:10:46.597 INFO:teuthology.orchestra.run.smithi027.stdout:(32/144): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 355 kB/s | 47 kB 00:00 2023-07-06T20:10:46.748 INFO:teuthology.orchestra.run.smithi027.stdout:(33/144): ceph-prometheus-alerts-18.0.0-4742.g7 97 kB/s | 15 kB 00:00 2023-07-06T20:10:47.190 INFO:teuthology.orchestra.run.smithi027.stdout:(34/144): ceph-volume-18.0.0-4742.g79365bb3.el8 585 kB/s | 258 kB 00:00 2023-07-06T20:10:47.431 INFO:teuthology.orchestra.run.smithi027.stdout:(35/144): cephadm-18.0.0-4742.g79365bb3.el8.noa 690 kB/s | 166 kB 00:00 2023-07-06T20:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout:(36/144): cephfs-top-18.0.0-4742.g79365bb3.el8. 180 kB/s | 18 kB 00:00 2023-07-06T20:10:47.832 INFO:teuthology.orchestra.run.smithi027.stdout:(37/144): automake-1.16.1-7.el8.noarch.rpm 2.3 MB/s | 713 kB 00:00 2023-07-06T20:10:47.907 INFO:teuthology.orchestra.run.smithi027.stdout:(38/144): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-07-06T20:10:47.958 INFO:teuthology.orchestra.run.smithi027.stdout:(39/144): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-06T20:10:48.016 INFO:teuthology.orchestra.run.smithi027.stdout:(40/144): jq-1.6-7.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-07-06T20:10:48.067 INFO:teuthology.orchestra.run.smithi027.stdout:(41/144): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-07-06T20:10:48.142 INFO:teuthology.orchestra.run.smithi027.stdout:(42/144): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2023-07-06T20:10:48.217 INFO:teuthology.orchestra.run.smithi027.stdout:(43/144): libtool-2.4.6-25.el8.x86_64.rpm 9.3 MB/s | 709 kB 00:00 2023-07-06T20:10:48.284 INFO:teuthology.orchestra.run.smithi027.stdout:(44/144): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-07-06T20:10:48.351 INFO:teuthology.orchestra.run.smithi027.stdout:(45/144): oniguruma-6.8.2-2.el8.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2023-07-06T20:10:48.552 INFO:teuthology.orchestra.run.smithi027.stdout:(46/144): openblas-0.3.15-4.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-07-06T20:10:48.752 INFO:teuthology.orchestra.run.smithi027.stdout:(47/144): openblas-threads-0.3.15-4.el8.x86_64. 24 MB/s | 4.9 MB 00:00 2023-07-06T20:10:48.852 INFO:teuthology.orchestra.run.smithi027.stdout:(48/144): postgresql-10.23-1.module_el8+300+2c5 16 MB/s | 1.6 MB 00:00 2023-07-06T20:10:48.936 INFO:teuthology.orchestra.run.smithi027.stdout:(49/144): postgresql-contrib-10.23-1.module_el8 11 MB/s | 903 kB 00:00 2023-07-06T20:10:49.170 INFO:teuthology.orchestra.run.smithi027.stdout:(50/144): postgresql-server-10.23-1.module_el8+ 22 MB/s | 5.1 MB 00:00 2023-07-06T20:10:49.403 INFO:teuthology.orchestra.run.smithi027.stdout:(51/144): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-07-06T20:10:49.471 INFO:teuthology.orchestra.run.smithi027.stdout:(52/144): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-07-06T20:10:49.521 INFO:teuthology.orchestra.run.smithi027.stdout:(53/144): python3-jsonpatch-1.21-2.el8.noarch.r 534 kB/s | 27 kB 00:00 2023-07-06T20:10:49.571 INFO:teuthology.orchestra.run.smithi027.stdout:(54/144): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-07-06T20:10:49.630 INFO:teuthology.orchestra.run.smithi027.stdout:(55/144): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-06T20:10:49.672 INFO:teuthology.orchestra.run.smithi027.stdout:(56/144): python3-markupsafe-0.23-19.el8.x86_64 933 kB/s | 39 kB 00:00 2023-07-06T20:10:49.897 INFO:teuthology.orchestra.run.smithi027.stdout:(57/144): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-07-06T20:10:49.972 INFO:teuthology.orchestra.run.smithi027.stdout:(58/144): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.9 MB/s | 225 kB 00:00 2023-07-06T20:10:50.023 INFO:teuthology.orchestra.run.smithi027.stdout:(59/144): python3-prettytable-0.7.2-14.el8.noar 882 kB/s | 44 kB 00:00 2023-07-06T20:10:50.073 INFO:teuthology.orchestra.run.smithi027.stdout:(60/144): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-06T20:10:50.140 INFO:teuthology.orchestra.run.smithi027.stdout:(61/144): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.8 MB/s | 126 kB 00:00 2023-07-06T20:10:50.190 INFO:teuthology.orchestra.run.smithi027.stdout:(62/144): python3-pyasn1-modules-0.3.7-6.el8.no 2.1 MB/s | 110 kB 00:00 2023-07-06T20:10:50.240 INFO:teuthology.orchestra.run.smithi027.stdout:(63/144): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-06T20:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout:(64/144): python3-scipy-1.0.0-21.module_el8.5.0 18 MB/s | 14 MB 00:00 2023-07-06T20:10:51.091 INFO:teuthology.orchestra.run.smithi027.stdout:(65/144): python3-werkzeug-0.12.2-4.el8.noarch. 6.0 MB/s | 457 kB 00:00 2023-07-06T20:10:51.158 INFO:teuthology.orchestra.run.smithi027.stdout:(66/144): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-07-06T20:10:51.208 INFO:teuthology.orchestra.run.smithi027.stdout:(67/144): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-07-06T20:10:51.259 INFO:teuthology.orchestra.run.smithi027.stdout:(68/144): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2023-07-06T20:10:51.359 INFO:teuthology.orchestra.run.smithi027.stdout:(69/144): ceph-mgr-diskprediction-local-18.0.0- 1.0 MB/s | 7.4 MB 00:07 2023-07-06T20:10:51.543 INFO:teuthology.orchestra.run.smithi027.stdout:(70/144): elfutils-libelf-devel-0.189-2.el8.x86 217 kB/s | 61 kB 00:00 2023-07-06T20:10:51.610 INFO:teuthology.orchestra.run.smithi027.stdout:(71/144): fuse-2.9.7-17.el8.x86_64.rpm 332 kB/s | 83 kB 00:00 2023-07-06T20:10:51.644 INFO:teuthology.orchestra.run.smithi027.stdout:(72/144): keyutils-libs-devel-1.5.10-9.el8.x86_ 479 kB/s | 48 kB 00:00 2023-07-06T20:10:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:(73/144): libaio-devel-0.3.112-1.el8.x86_64.rpm 225 kB/s | 19 kB 00:00 2023-07-06T20:10:51.835 INFO:teuthology.orchestra.run.smithi027.stdout:(74/144): libcom_err-devel-1.45.6-5.el8.x86_64. 359 kB/s | 39 kB 00:00 2023-07-06T20:10:51.861 INFO:teuthology.orchestra.run.smithi027.stdout:(75/144): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.2 MB/s | 561 kB 00:00 2023-07-06T20:10:51.961 INFO:teuthology.orchestra.run.smithi027.stdout:(76/144): libquadmath-8.5.0-20.el8.x86_64.rpm 1.7 MB/s | 172 kB 00:00 2023-07-06T20:10:52.037 INFO:teuthology.orchestra.run.smithi027.stdout:(77/144): librabbitmq-0.9.0-4.el8.x86_64.rpm 626 kB/s | 47 kB 00:00 2023-07-06T20:10:52.137 INFO:teuthology.orchestra.run.smithi027.stdout:(78/144): libselinux-devel-2.9-8.el8.x86_64.rpm 2.0 MB/s | 200 kB 00:00 2023-07-06T20:10:52.196 INFO:teuthology.orchestra.run.smithi027.stdout:(79/144): libgfortran-8.5.0-20.el8.x86_64.rpm 1.8 MB/s | 645 kB 00:00 2023-07-06T20:10:52.230 INFO:teuthology.orchestra.run.smithi027.stdout:(80/144): libsepol-devel-2.9-3.el8.x86_64.rpm 946 kB/s | 87 kB 00:00 2023-07-06T20:10:52.280 INFO:teuthology.orchestra.run.smithi027.stdout:(81/144): libuuid-devel-2.32.1-42.el8.x86_64.rp 1.2 MB/s | 99 kB 00:00 2023-07-06T20:10:52.305 INFO:teuthology.orchestra.run.smithi027.stdout:(82/144): libverto-devel-0.3.2-2.el8.x86_64.rpm 240 kB/s | 18 kB 00:00 2023-07-06T20:10:52.347 INFO:teuthology.orchestra.run.smithi027.stdout:(83/144): libzstd-devel-1.4.4-1.el8.x86_64.rpm 656 kB/s | 44 kB 00:00 2023-07-06T20:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout:(84/144): pcre2-devel-10.32-3.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-07-06T20:10:52.598 INFO:teuthology.orchestra.run.smithi027.stdout:(85/144): openssl-devel-1.1.1k-9.el8.x86_64.rpm 11 MB/s | 3.2 MB 00:00 2023-07-06T20:10:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:(86/144): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-07-06T20:10:52.682 INFO:teuthology.orchestra.run.smithi027.stdout:(87/144): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-07-06T20:10:52.724 INFO:teuthology.orchestra.run.smithi027.stdout:(88/144): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.4 MB/s | 249 kB 00:00 2023-07-06T20:10:52.782 INFO:teuthology.orchestra.run.smithi027.stdout:(89/144): python3-cryptography-3.2.1-6.el8.x86_ 6.2 MB/s | 641 kB 00:00 2023-07-06T20:10:52.799 INFO:teuthology.orchestra.run.smithi027.stdout:(90/144): python3-jwt-1.6.1-2.el8.noarch.rpm 568 kB/s | 43 kB 00:00 2023-07-06T20:10:52.858 INFO:teuthology.orchestra.run.smithi027.stdout:(91/144): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2023-07-06T20:10:52.900 INFO:teuthology.orchestra.run.smithi027.stdout:(92/144): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2023-07-06T20:10:52.925 INFO:teuthology.orchestra.run.smithi027.stdout:(93/144): python3-requests-oauthlib-1.0.0-1.el8 640 kB/s | 43 kB 00:00 2023-07-06T20:10:52.983 INFO:teuthology.orchestra.run.smithi027.stdout:(94/144): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 894 kB/s | 74 kB 00:00 2023-07-06T20:10:53.017 INFO:teuthology.orchestra.run.smithi027.stdout:(95/144): zlib-devel-1.2.11-21.el8.x86_64.rpm 632 kB/s | 58 kB 00:00 2023-07-06T20:10:53.459 INFO:teuthology.orchestra.run.smithi027.stdout:(96/144): liboath-2.6.2-3.el8.x86_64.rpm 133 kB/s | 59 kB 00:00 2023-07-06T20:10:53.560 INFO:teuthology.orchestra.run.smithi027.stdout:(97/144): libunwind-1.3.1-3.el8.x86_64.rpm 752 kB/s | 75 kB 00:00 2023-07-06T20:10:53.652 INFO:teuthology.orchestra.run.smithi027.stdout:(98/144): gperftools-libs-2.7-9.el8.x86_64.rpm 459 kB/s | 306 kB 00:00 2023-07-06T20:10:53.735 INFO:teuthology.orchestra.run.smithi027.stdout:(99/144): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2023-07-06T20:10:53.902 INFO:teuthology.orchestra.run.smithi027.stdout:(100/144): python3-asyncssh-2.7.0-2.el8.noarch. 1.4 MB/s | 476 kB 00:00 2023-07-06T20:10:53.928 INFO:teuthology.orchestra.run.smithi027.stdout:(101/144): python3-beautifulsoup4-4.6.3-2.el8.1 965 kB/s | 185 kB 00:00 2023-07-06T20:10:53.977 INFO:teuthology.orchestra.run.smithi027.stdout:(102/144): python3-cachetools-3.1.1-4.el8.noarc 439 kB/s | 33 kB 00:00 2023-07-06T20:10:54.003 INFO:teuthology.orchestra.run.smithi027.stdout:(103/144): python3-certifi-2018.10.15-7.el8.noa 217 kB/s | 16 kB 00:00 2023-07-06T20:10:54.463 INFO:teuthology.orchestra.run.smithi027.stdout:(104/144): python3-cheroot-8.5.2-1.el8.noarch.r 356 kB/s | 173 kB 00:00 2023-07-06T20:10:54.504 INFO:teuthology.orchestra.run.smithi027.stdout:(105/144): python3-cherrypy-18.4.0-1.el8.noarch 768 kB/s | 384 kB 00:00 2023-07-06T20:10:54.604 INFO:teuthology.orchestra.run.smithi027.stdout:(106/144): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-07-06T20:10:54.630 INFO:teuthology.orchestra.run.smithi027.stdout:(107/144): python3-google-auth-1.1.1-10.el8.noa 490 kB/s | 81 kB 00:00 2023-07-06T20:10:54.705 INFO:teuthology.orchestra.run.smithi027.stdout:(108/144): python3-jaraco-functools-2.0-4.el8.n 176 kB/s | 18 kB 00:00 2023-07-06T20:10:54.863 INFO:teuthology.orchestra.run.smithi027.stdout:(109/144): python3-logutils-0.3.5-11.el8.noarch 308 kB/s | 49 kB 00:00 2023-07-06T20:10:55.014 INFO:teuthology.orchestra.run.smithi027.stdout:(110/144): python3-more-itertools-7.2.0-3.el8.n 392 kB/s | 59 kB 00:00 2023-07-06T20:10:55.155 INFO:teuthology.orchestra.run.smithi027.stdout:(111/144): python3-natsort-7.1.1-2.el8.noarch.r 426 kB/s | 60 kB 00:00 2023-07-06T20:10:55.573 INFO:teuthology.orchestra.run.smithi027.stdout:(112/144): python3-pecan-1.3.2-9.el8.noarch.rpm 679 kB/s | 283 kB 00:00 2023-07-06T20:10:55.656 INFO:teuthology.orchestra.run.smithi027.stdout:(113/144): python3-portend-2.6-1.el8.noarch.rpm 188 kB/s | 16 kB 00:00 2023-07-06T20:10:55.748 INFO:teuthology.orchestra.run.smithi027.stdout:(114/144): python3-repoze-lru-0.7-6.el8.noarch. 367 kB/s | 34 kB 00:00 2023-07-06T20:10:55.998 INFO:teuthology.orchestra.run.smithi027.stdout:(115/144): python3-routes-2.4.1-12.el8.noarch.r 787 kB/s | 196 kB 00:00 2023-07-06T20:10:56.149 INFO:teuthology.orchestra.run.smithi027.stdout:(116/144): python3-kubernetes-11.0.0-6.el8.noar 786 kB/s | 1.2 MB 00:01 2023-07-06T20:10:56.174 INFO:teuthology.orchestra.run.smithi027.stdout:(117/144): python3-rsa-4.8-1.el8.noarch.rpm 351 kB/s | 61 kB 00:00 2023-07-06T20:10:56.208 INFO:teuthology.orchestra.run.smithi027.stdout:(118/144): python3-simplegeneric-0.8.1-17.el8.n 319 kB/s | 19 kB 00:00 2023-07-06T20:10:56.233 INFO:teuthology.orchestra.run.smithi027.stdout:(119/144): python3-singledispatch-3.4.0.3-18.el 408 kB/s | 24 kB 00:00 2023-07-06T20:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout:(120/144): python3-tempora-1.14.1-5.el8.noarch. 499 kB/s | 29 kB 00:00 2023-07-06T20:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout:(121/144): python3-trustme-0.6.0-4.el8.noarch.r 470 kB/s | 27 kB 00:00 2023-07-06T20:10:56.876 INFO:teuthology.orchestra.run.smithi027.stdout:(122/144): python3-webob-1.8.5-1.el8.1.noarch.r 431 kB/s | 251 kB 00:00 2023-07-06T20:10:57.101 INFO:teuthology.orchestra.run.smithi027.stdout:(123/144): python3-waitress-1.4.3-1.el8.noarch. 306 kB/s | 255 kB 00:00 2023-07-06T20:10:57.334 INFO:teuthology.orchestra.run.smithi027.stdout:(124/144): python3-websocket-client-0.56.0-5.el 133 kB/s | 61 kB 00:00 2023-07-06T20:10:57.418 INFO:teuthology.orchestra.run.smithi027.stdout:(125/144): python3-webtest-2.0.33-1.el8.noarch. 269 kB/s | 85 kB 00:00 2023-07-06T20:10:57.560 INFO:teuthology.orchestra.run.smithi027.stdout:(126/144): python3-zc-lockfile-2.0-2.el8.noarch 102 kB/s | 23 kB 00:00 2023-07-06T20:11:00.219 INFO:teuthology.orchestra.run.smithi027.stdout:(127/144): librados2-18.0.0-4742.g79365bb3.el8. 1.4 MB/s | 3.7 MB 00:02 2023-07-06T20:11:01.627 INFO:teuthology.orchestra.run.smithi027.stdout:(128/144): thrift-0.13.0-2.el8.x86_64.rpm 422 kB/s | 1.7 MB 00:04 2023-07-06T20:11:01.861 INFO:teuthology.orchestra.run.smithi027.stdout:(129/144): librbd1-18.0.0-4742.g79365bb3.el8.x8 2.2 MB/s | 3.6 MB 00:01 2023-07-06T20:11:01.936 INFO:teuthology.orchestra.run.smithi027.stdout:(130/144): NetworkManager-initscripts-updown-1. 465 kB/s | 143 kB 00:00 2023-07-06T20:11:02.078 INFO:teuthology.orchestra.run.smithi027.stdout:(131/144): elfutils-libelf-0.189-2.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2023-07-06T20:11:02.103 INFO:teuthology.orchestra.run.smithi027.stdout:(132/144): elfutils-debuginfod-client-0.189-2.e 313 kB/s | 76 kB 00:00 2023-07-06T20:11:02.187 INFO:teuthology.orchestra.run.smithi027.stdout:(133/144): elfutils-libs-0.189-2.el8.x86_64.rpm 2.7 MB/s | 302 kB 00:00 2023-07-06T20:11:02.271 INFO:teuthology.orchestra.run.smithi027.stdout:(134/144): libblkid-2.32.1-42.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-07-06T20:11:02.371 INFO:teuthology.orchestra.run.smithi027.stdout:(135/144): libfdisk-2.32.1-42.el8.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2023-07-06T20:11:02.405 INFO:teuthology.orchestra.run.smithi027.stdout:(136/144): iproute-6.2.0-2.el8.x86_64.rpm 2.9 MB/s | 880 kB 00:00 2023-07-06T20:11:02.447 INFO:teuthology.orchestra.run.smithi027.stdout:(137/144): libmount-2.32.1-42.el8.x86_64.rpm 3.1 MB/s | 236 kB 00:00 2023-07-06T20:11:02.489 INFO:teuthology.orchestra.run.smithi027.stdout:(138/144): libsmartcols-2.32.1-42.el8.x86_64.rp 2.1 MB/s | 179 kB 00:00 2023-07-06T20:11:02.531 INFO:teuthology.orchestra.run.smithi027.stdout:(139/144): libuuid-2.32.1-42.el8.x86_64.rpm 1.1 MB/s | 98 kB 00:00 2023-07-06T20:11:02.614 INFO:teuthology.orchestra.run.smithi027.stdout:(140/144): util-linux-user-2.32.1-42.el8.x86_64 1.2 MB/s | 102 kB 00:00 2023-07-06T20:11:02.723 INFO:teuthology.orchestra.run.smithi027.stdout:(141/144): xfsdump-3.1.8-6.el8.x86_64.rpm 3.0 MB/s | 334 kB 00:00 2023-07-06T20:11:02.773 INFO:teuthology.orchestra.run.smithi027.stdout:(142/144): util-linux-2.32.1-42.el8.x86_64.rpm 9.1 MB/s | 2.6 MB 00:00 2023-07-06T20:11:02.882 INFO:teuthology.orchestra.run.smithi027.stdout:(143/144): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.9 MB/s | 1.1 MB 00:00 2023-07-06T20:11:05.790 INFO:teuthology.orchestra.run.smithi027.stdout:(144/144): ceph-test-18.0.0-4742.g79365bb3.el8. 1.1 MB/s | 48 MB 00:42 2023-07-06T20:11:05.793 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:11:05.794 INFO:teuthology.orchestra.run.smithi027.stdout:Total 4.3 MB/s | 212 MB 00:48 2023-07-06T20:11:07.655 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-07-06T20:11:08.103 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-07-06T20:11:08.103 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-07-06T20:11:09.613 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-07-06T20:11:09.617 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-07-06T20:11:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-07-06T20:11:15.213 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-07-06T20:11:15.401 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/160 2023-07-06T20:11:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T20:11:15.642 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T20:11:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T20:11:15.899 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T20:11:16.124 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/160 2023-07-06T20:11:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.189-2.el8.x86_64 5/160 2023-07-06T20:11:16.581 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/160 2023-07-06T20:11:16.745 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T20:11:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T20:11:17.445 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T20:11:17.924 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T20:11:18.128 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T20:11:18.269 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T20:11:18.478 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T20:11:18.587 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T20:11:18.806 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T20:11:18.970 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T20:11:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T20:11:19.463 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T20:11:19.557 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T20:11:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/160 2023-07-06T20:11:19.973 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/160 2023-07-06T20:11:20.433 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpq-13.5-1.el8.x86_64 16/160 2023-07-06T20:11:20.625 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/160 2023-07-06T20:11:20.761 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T20:11:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T20:11:21.036 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T20:11:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T20:11:21.388 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/160 2023-07-06T20:11:21.557 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/160 2023-07-06T20:11:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T20:11:21.851 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T20:11:22.003 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T20:11:22.752 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T20:11:22.952 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T20:11:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T20:11:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T20:11:24.796 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T20:11:25.145 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/160 2023-07-06T20:11:25.371 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/160 2023-07-06T20:11:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/160 2023-07-06T20:11:25.760 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/160 2023-07-06T20:11:25.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 30/160 2023-07-06T20:11:26.134 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 31/160 2023-07-06T20:11:26.375 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 32/160 2023-07-06T20:11:27.521 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/160 2023-07-06T20:11:27.878 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/160 2023-07-06T20:11:28.169 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/160 2023-07-06T20:11:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T20:11:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T20:11:30.960 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/160 2023-07-06T20:11:31.163 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/160 2023-07-06T20:11:31.417 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/160 2023-07-06T20:11:31.618 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 40/160 2023-07-06T20:11:31.806 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 41/160 2023-07-06T20:11:31.998 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.189-2.el8.x86_64 42/160 2023-07-06T20:11:32.171 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-debuginfod-client-0.189-2.el8.x86_64 43/160 2023-07-06T20:11:32.401 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T20:11:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T20:11:32.743 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T20:11:32.835 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T20:11:32.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.32.1-42.el8.x86_64 46/160 2023-07-06T20:11:33.193 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T20:11:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T20:11:34.077 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T20:11:34.107 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T20:11:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/160 2023-07-06T20:11:34.753 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/160 2023-07-06T20:11:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/160 2023-07-06T20:11:35.166 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/160 2023-07-06T20:11:35.316 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/160 2023-07-06T20:11:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/160 2023-07-06T20:11:35.728 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/160 2023-07-06T20:11:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/160 2023-07-06T20:11:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/160 2023-07-06T20:11:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/160 2023-07-06T20:11:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/160 2023-07-06T20:11:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/160 2023-07-06T20:11:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/160 2023-07-06T20:11:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/160 2023-07-06T20:11:37.343 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/160 2023-07-06T20:11:37.639 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/160 2023-07-06T20:11:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/160 2023-07-06T20:11:38.375 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/160 2023-07-06T20:11:38.653 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/160 2023-07-06T20:11:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/160 2023-07-06T20:11:39.738 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/160 2023-07-06T20:11:39.942 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T20:11:40.023 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T20:11:40.231 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 71/160 2023-07-06T20:11:40.466 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 72/160 2023-07-06T20:11:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 73/160 2023-07-06T20:11:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 74/160 2023-07-06T20:11:41.534 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 75/160 2023-07-06T20:11:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 76/160 2023-07-06T20:11:42.012 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 77/160 2023-07-06T20:11:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/160 2023-07-06T20:11:43.791 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/160 2023-07-06T20:11:43.986 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 80/160 2023-07-06T20:11:44.235 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/160 2023-07-06T20:11:44.894 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/160 2023-07-06T20:11:45.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/160 2023-07-06T20:11:45.356 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/160 2023-07-06T20:11:45.578 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/160 2023-07-06T20:11:45.939 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/160 2023-07-06T20:11:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/160 2023-07-06T20:11:46.311 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/160 2023-07-06T20:11:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/160 2023-07-06T20:11:46.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/160 2023-07-06T20:11:47.063 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/160 2023-07-06T20:11:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/160 2023-07-06T20:11:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/160 2023-07-06T20:11:47.645 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/160 2023-07-06T20:11:47.862 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/160 2023-07-06T20:11:48.053 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/160 2023-07-06T20:11:48.233 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T20:11:48.360 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T20:11:48.569 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 98/160 2023-07-06T20:11:48.725 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 99/160 2023-07-06T20:11:51.737 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T20:11:51.983 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T20:11:52.726 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T20:11:53.046 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T20:11:53.245 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T20:11:53.449 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T20:12:14.834 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /run 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2023-07-06T20:12:14.835 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2023-07-06T20:12:14.836 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2023-07-06T20:12:14.836 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2023-07-06T20:12:14.836 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:15.238 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T20:12:17.096 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T20:12:17.351 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T20:12:17.634 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T20:12:17.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T20:12:18.448 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T20:12:18.646 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T20:12:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T20:12:18.905 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 107/160 2023-07-06T20:12:19.111 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 108/160 2023-07-06T20:12:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T20:12:19.341 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T20:12:19.650 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.6-7.el8.x86_64 110/160 2023-07-06T20:12:19.887 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.1-7.el8.noarch 111/160 2023-07-06T20:12:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T20:12:20.663 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T20:12:20.795 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 113/160 2023-07-06T20:12:21.021 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 114/160 2023-07-06T20:12:21.245 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T20:12:21.520 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T20:12:21.742 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T20:12:22.935 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T20:12:23.112 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T20:12:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T20:12:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 118/160 2023-07-06T20:12:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T20:12:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T20:12:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T20:12:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T20:12:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 121/160 2023-07-06T20:12:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T20:12:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T20:12:31.446 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 123/160 2023-07-06T20:12:31.758 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T20:12:31.832 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T20:12:31.969 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T20:12:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T20:12:32.236 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T20:12:32.672 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T20:12:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T20:12:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T20:12:33.548 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 128/160 2023-07-06T20:12:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T20:12:35.133 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 130/160 2023-07-06T20:12:35.336 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.189-2.el8.x86_64 131/160 2023-07-06T20:12:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-user-2.32.1-42.el8.x86_64 132/160 2023-07-06T20:12:35.733 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/160 2023-07-06T20:12:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 134/160 2023-07-06T20:12:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T20:12:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T20:12:37.582 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T20:12:37.966 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 136/160 2023-07-06T20:12:38.228 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 137/160 2023-07-06T20:12:38.437 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 138/160 2023-07-06T20:12:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 139/160 2023-07-06T20:12:39.054 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 140/160 2023-07-06T20:12:39.226 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T20:12:39.256 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T20:12:39.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 142/160 2023-07-06T20:12:39.776 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T20:12:39.917 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T20:12:40.135 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T20:12:40.162 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T20:12:40.345 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 145/160 2023-07-06T20:12:40.526 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 146/160 2023-07-06T20:12:40.698 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 147/160 2023-07-06T20:12:40.699 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T20:12:40.981 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T20:12:40.982 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T20:12:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T20:12:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T20:12:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T20:12:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T20:12:41.644 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T20:12:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/160 2023-07-06T20:12:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/160 2023-07-06T20:12:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T20:12:42.170 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T20:12:42.396 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T20:12:42.397 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T20:12:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T20:12:42.613 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T20:12:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T20:12:42.838 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T20:12:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T20:12:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T20:12:43.288 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T20:12:43.478 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T20:12:43.680 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T20:12:43.680 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T20:12:43.680 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/160 2023-07-06T20:12:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/160 2023-07-06T20:12:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/160 2023-07-06T20:12:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/160 2023-07-06T20:12:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/160 2023-07-06T20:12:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/160 2023-07-06T20:12:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/160 2023-07-06T20:12:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T20:12:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T20:12:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/160 2023-07-06T20:12:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/160 2023-07-06T20:12:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/160 2023-07-06T20:12:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/160 2023-07-06T20:12:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/160 2023-07-06T20:12:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.6-7.el8.x86_64 42/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/160 2023-07-06T20:12:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/160 2023-07-06T20:12:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/160 2023-07-06T20:12:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.189-2.el8.x86_64 71/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/160 2023-07-06T20:12:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 77/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 78/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/160 2023-07-06T20:12:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.32.1-42.el8.x86_64 82/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 85/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/160 2023-07-06T20:12:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/160 2023-07-06T20:12:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/160 2023-07-06T20:12:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/160 2023-07-06T20:12:44.449 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 118/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/160 2023-07-06T20:12:44.450 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/160 2023-07-06T20:12:44.451 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-8.el 133/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.189-2.el8.x86_64 135/160 2023-07-06T20:12:44.452 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 136/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-2.el8.x86_64 137/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 138/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-2.el8.x86_64 139/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 140/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 141/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 142/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.32.1-42.el8.x86_64 143/160 2023-07-06T20:12:44.453 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 144/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.32.1-42.el8.x86_64 145/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 146/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.32.1-42.el8.x86_64 147/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 148/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.32.1-42.el8.x86_64 149/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 150/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.32.1-42.el8.x86_64 151/160 2023-07-06T20:12:44.454 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 152/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.32.1-42.el8.x86_64 153/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.32.1-42.el8.x86_64 155/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 157/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 158/160 2023-07-06T20:12:44.455 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 159/160 2023-07-06T20:12:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 160/160 2023-07-06T20:12:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:46.580 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-07-06T20:12:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-initscripts-updown-1:1.40.16-8.el8.noarch 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client-0.189-2.el8.x86_64 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.189-2.el8.x86_64 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.189-2.el8.x86_64 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: iproute-6.2.0-2.el8.x86_64 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: libmount-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-07-06T20:12:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.189-2.el8.x86_64 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.7-17.el8.x86_64 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T20:12:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.6-7.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-07-06T20:12:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.32.1-42.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-07-06T20:12:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.15-4.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-07-06T20:12:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T20:12:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T20:12:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T20:12:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.0.6-14.el8.noarch 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T20:12:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T20:12:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T20:12:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T20:12:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T20:12:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T20:12:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T20:12:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:12:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T20:12:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T20:12:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T20:12:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-07-06T20:12:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T20:12:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-07-06T20:12:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:46.601 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-07-06T20:12:46.884 DEBUG:teuthology.parallel:result is None 2023-07-06T20:12:46.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:12:47.024 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T20:12:47.082 INFO:teuthology.orchestra.run.smithi027.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T20:12:47.082 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T20:12:47.083 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T20:12:47.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:12:47.432 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T20:12:47.494 INFO:teuthology.orchestra.run.smithi153.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T20:12:47.494 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T20:12:47.495 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T20:12:47.495 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-06T20:12:47.496 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:47.496 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T20:12:47.575 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:47.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T20:12:47.651 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-06T20:12:47.652 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:47.652 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T20:12:47.725 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T20:12:47.836 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:47.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T20:12:47.905 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T20:12:48.013 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-06T20:12:48.014 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:48.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T20:12:48.087 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T20:12:48.199 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:48.200 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T20:12:48.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T20:12:48.374 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-06T20:12:48.375 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:48.375 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T20:12:48.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T20:12:48.522 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:48.522 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T20:12:48.589 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T20:12:48.697 INFO:teuthology.run_tasks:Running task ceph... 2023-07-06T20:12:48.765 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-06T20:12:48.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2023-07-06T20:12:48.770 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /var/log/ceph 2023-07-06T20:12:48.839 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-06T20:12:48.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T20:12:48.843 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T20:12:48.913 INFO:tasks.ceph:Creating extra log directories... 2023-07-06T20:12:48.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T20:12:48.918 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T20:12:48.993 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-06T20:12:48.994 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-06T20:12:48.994 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-06T20:12:48.994 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:48.994 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T20:12:49.043 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T20:12:49.601 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T20:12:49.676 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:49.677 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T20:12:49.723 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T20:12:49.935 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T20:12:50.009 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-06T20:12:50.010 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-06T20:12:50.010 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147', 'branch': 'wip-rishabh-2023Jul5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}', 'email': None, 'first_in_suite': False, 'job_id': '7328147', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul5', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi027.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvVqYBqk6RwHd+gGqqcWtRbd74UJHsWYb4z5uVCfrZeMRhCv2tuwoMsQ044WHvlZMgS/Qazj6ja8+mE7864w35fl/CNZkDN3P4U7bwP8AAD5ZLAfagzgMFWLN+x0CLjKkyZB01iBc/aLgTvrtdvqaAHX9ifM1TVEo6/HsLd6lq/LUHz7VsJrZljDpyCDihvwM6FbCSlOS8UyiaNuJRAWX1YY+8QSmoX8EzVIgRh+6ZX2RlY7FIKC+WgsGT9/OvRIQ1T2Tjv0oOXfnyp2rghgplKgOX4Yk2mVWQF6QMHakH8r6DPwVRkgwOP33YYz5ljGabYsFdPkC8xgqtl1tetKgV1kPwM1iikRpfoTnyS53JiEyu5h1HNo+khB3DFNCqZcdl0eG8gsAJ2tXXWPCVa+Pz9F9xbfZzfM/OQsk0Grb6Xsv5y4Md/AN/qaK/JDSpho2W9QtQ296ujy37u7ufg/gagH2amLfwuHcvgQXIgs6G+sB2/CEDbTzX3qf4NKad3zs=', 'smithi153.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbolPH+s+kUvv9GKmqZ24iklGKuOHmgaxXHyTohmTDlgsm5f43LwCq1OPDjKu6ZcN9ZRN24+VCCBQGETHv2hQFBvv2ut98zW19sVLuzpj1t8CFfr+1i2ZrRuU/NdFNuntkay/Z47cfg6ad7qIVH/EV9YA0G6XDCfHVN58isouGsLxWP3EmLDZnwxyLgGMUYn9I11qtLHCTu5Ru5lPUcbYuFm7I1NLiGSJU85I/lIjLEURU1awEB11ANyeBFIeZcr/+UawngE76MtYinp5PfiBKD/RXagAVXNioeD2CHpqqmKU/SfXK+ufDMby/PjnWL+lp+1wZ9b1Oqv0IXYJ5YTIk/kDdV9OPtK0CAGP3Dnn3Wf6optYpEsqpHsMbPaEF1xotK6ZVslPxNHpcPKYaRpWOiTOydPnvgIbttvBjKxzG21P6LSt/5YtikruWUfCUAzz8xbMNb7nWsACYRbkJAcmqkShJvYMuIMqI+PgV9tsN0fjSMR65yeYadmWpb+1LVTs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_quota']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b53f236c1077f1d295baafb9e0ca9a4b447019d7', 'timestamp': '2023-07-06_19:33:28', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-06T20:12:50.011 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T20:12:50.015 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T20:12:50.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T20:12:50.066 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T20:12:50.170 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:50.170 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T20:12:50.216 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T20:12:50.217 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2023-07-06T20:12:50.306 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 444 Links: 1 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-07-06 20:12:43.909720067 +0000 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-07-06 20:04:19.697399937 +0000 2023-07-06T20:12:50.307 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-07-06 20:04:19.697399937 +0000 2023-07-06T20:12:50.308 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-07-06 20:04:19.666400711 +0000 2023-07-06T20:12:50.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T20:12:50.375 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-07-06T20:12:50.376 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-07-06T20:12:50.376 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000279602 s, 1.8 MB/s 2023-07-06T20:12:50.378 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T20:12:50.429 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2023-07-06T20:12:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T20:12:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:50.476 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-06T20:12:50.476 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:50.476 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:50.476 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-07-06 20:12:43.910720041 +0000 2023-07-06T20:12:50.477 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-07-06 20:04:20.576378005 +0000 2023-07-06T20:12:50.477 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-07-06 20:04:20.576378005 +0000 2023-07-06T20:12:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-07-06 20:04:20.547378728 +0000 2023-07-06T20:12:50.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T20:12:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-07-06T20:12:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-07-06T20:12:50.587 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000291473 s, 1.8 MB/s 2023-07-06T20:12:50.589 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T20:12:50.640 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2023-07-06T20:12:50.688 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T20:12:50.688 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:50.689 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-06T20:12:50.689 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:50.689 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:50.689 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-07-06 20:12:43.910720041 +0000 2023-07-06T20:12:50.690 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-07-06 20:04:21.457356022 +0000 2023-07-06T20:12:50.690 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-07-06 20:04:21.457356022 +0000 2023-07-06T20:12:50.690 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-07-06 20:04:21.429356721 +0000 2023-07-06T20:12:50.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T20:12:50.761 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-07-06T20:12:50.761 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-07-06T20:12:50.761 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000284981 s, 1.8 MB/s 2023-07-06T20:12:50.763 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T20:12:50.816 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2023-07-06T20:12:50.864 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T20:12:50.865 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:50.865 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-06T20:12:50.865 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:50.865 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:50.865 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-07-06 20:12:43.910720041 +0000 2023-07-06T20:12:50.866 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-07-06 20:04:22.319334514 +0000 2023-07-06T20:12:50.866 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-07-06 20:04:22.319334514 +0000 2023-07-06T20:12:50.866 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-07-06 20:04:22.294335137 +0000 2023-07-06T20:12:50.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T20:12:50.935 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-07-06T20:12:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-07-06T20:12:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000290242 s, 1.8 MB/s 2023-07-06T20:12:50.938 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T20:12:50.989 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T20:12:50.990 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:50.990 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T20:12:51.038 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T20:12:51.038 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2023-07-06T20:12:51.125 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T20:12:51.125 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:51.125 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 444 Links: 1 2023-07-06T20:12:51.126 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:51.126 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:51.126 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-07-06 20:07:37.454270498 +0000 2023-07-06T20:12:51.126 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-07-06 20:04:11.121443296 +0000 2023-07-06T20:12:51.126 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-07-06 20:04:11.121443296 +0000 2023-07-06T20:12:51.126 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2023-07-06 20:04:11.096443735 +0000 2023-07-06T20:12:51.127 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T20:12:51.193 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-07-06T20:12:51.193 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-07-06T20:12:51.193 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000466509 s, 1.1 MB/s 2023-07-06T20:12:51.195 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T20:12:51.243 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2023-07-06T20:12:51.289 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T20:12:51.289 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:51.289 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-06T20:12:51.289 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:51.289 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:51.290 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-07-06 20:07:37.455270480 +0000 2023-07-06T20:12:51.290 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-07-06 20:04:11.972428356 +0000 2023-07-06T20:12:51.290 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-07-06 20:04:11.972428356 +0000 2023-07-06T20:12:51.290 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2023-07-06 20:04:11.947428795 +0000 2023-07-06T20:12:51.290 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T20:12:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-07-06T20:12:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-07-06T20:12:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000312327 s, 1.6 MB/s 2023-07-06T20:12:51.359 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T20:12:51.407 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2023-07-06T20:12:51.452 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T20:12:51.452 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:51.452 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-06T20:12:51.452 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:51.453 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:51.453 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-07-06 20:07:37.455270480 +0000 2023-07-06T20:12:51.453 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-07-06 20:04:12.843413064 +0000 2023-07-06T20:12:51.453 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-07-06 20:04:12.843413064 +0000 2023-07-06T20:12:51.453 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2023-07-06 20:04:12.818413503 +0000 2023-07-06T20:12:51.453 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T20:12:51.558 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-07-06T20:12:51.558 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-07-06T20:12:51.558 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000283355 s, 1.8 MB/s 2023-07-06T20:12:51.560 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T20:12:51.607 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2023-07-06T20:12:51.652 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T20:12:51.653 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:51.653 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-06T20:12:51.653 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:51.653 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:51.653 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-07-06 20:07:37.455270480 +0000 2023-07-06T20:12:51.653 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-07-06 20:04:13.711397826 +0000 2023-07-06T20:12:51.654 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-07-06 20:04:13.711397826 +0000 2023-07-06T20:12:51.654 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2023-07-06 20:04:13.686398265 +0000 2023-07-06T20:12:51.654 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T20:12:51.757 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-07-06T20:12:51.757 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-07-06T20:12:51.758 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.00030082 s, 1.7 MB/s 2023-07-06T20:12:51.759 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T20:12:51.806 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T20:12:51.807 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T20:12:51.807 INFO:tasks.ceph:Generating config... 2023-07-06T20:12:51.808 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-06T20:12:51.808 INFO:tasks.ceph:[client] debug client = 20 2023-07-06T20:12:51.809 INFO:tasks.ceph:[client] debug ms = 1 2023-07-06T20:12:51.809 INFO:tasks.ceph:[client] pid file = 2023-07-06T20:12:51.809 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-06T20:12:51.809 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-06T20:12:51.810 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-07-06T20:12:51.810 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-06T20:12:51.810 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-06T20:12:51.810 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-06T20:12:51.810 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-06T20:12:51.811 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-06T20:12:51.811 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-06T20:12:51.811 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-06T20:12:51.811 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-06T20:12:51.811 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-06T20:12:51.812 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-06T20:12:51.812 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-06T20:12:51.812 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-06T20:12:51.812 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-06T20:12:51.812 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-06T20:12:51.813 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-06T20:12:51.813 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-06T20:12:51.813 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-06T20:12:51.813 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-06T20:12:51.813 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-07-06T20:12:51.814 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-06T20:12:51.814 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-06T20:12:51.814 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-07-06T20:12:51.814 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-07-06T20:12:51.814 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-06T20:12:51.815 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-06T20:12:51.815 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-07-06T20:12:51.815 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-06T20:12:51.815 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-06T20:12:51.815 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-06T20:12:51.816 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-06T20:12:51.816 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-06T20:12:51.816 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-06T20:12:51.816 INFO:tasks.ceph:Setting up mon.a... 2023-07-06T20:12:51.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-06T20:12:51.934 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2023-07-06T20:12:51.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-06T20:12:52.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T20:12:52.118 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.b', '172.21.15.153'), ('mon.c', '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]')] 2023-07-06T20:12:52.118 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-06T20:12:52.119 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:52.120 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T20:12:52.167 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --add b 172.21.15.153 --addv c '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:12:52.300 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:12:52.300 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid c2a9da8e-f30b-43f7-b235-3303ae838054 2023-07-06T20:12:52.300 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = pacific 2023-07-06T20:12:52.300 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2023-07-06T20:12:52.300 INFO:teuthology.orchestra.run.smithi027.stdout:fsid c2a9da8e-f30b-43f7-b235-3303ae838054 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2023-07-06T20:12:52.299396+0000 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:created 2023-07-06T20:12:52.299396+0000 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 16 (pacific) 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2023-07-06T20:12:52.301 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] mon.c 2023-07-06T20:12:52.302 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-06T20:12:52.303 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T20:12:52.350 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c2a9da8e-f30b-43f7-b235-3303ae838054... 2023-07-06T20:12:52.351 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T20:12:52.356 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T20:12:52.472 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.473 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.474 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T20:12:52.475 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-07-06T20:12:52.475 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.475 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T20:12:52.475 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T20:12:52.475 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.475 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-07-06T20:12:52.476 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.476 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-07-06T20:12:52.476 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-07-06T20:12:52.476 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-07-06T20:12:52.476 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-07-06T20:12:52.476 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T20:12:52.477 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-07-06T20:12:52.478 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.27,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on skipped message = False 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = c2a9da8e-f30b-43f7-b235-3303ae838054 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.479 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-07-06T20:12:52.480 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.481 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-07-06T20:12:52.482 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-07-06T20:12:52.483 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-07-06T20:12:52.484 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-07-06T20:12:52.485 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-07-06T20:12:52.486 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-07-06T20:12:52.486 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-07-06T20:12:52.486 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-07-06T20:12:52.486 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.486 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-07-06T20:12:52.487 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T20:12:52.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.489 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T20:12:52.490 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T20:12:52.491 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-07-06T20:12:52.491 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-07-06T20:12:52.491 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-07-06T20:12:52.491 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-07-06T20:12:52.491 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-07-06T20:12:52.491 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-07-06T20:12:52.492 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds balancer = 20 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-07-06T20:12:52.493 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-07-06T20:12:52.494 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-07-06T20:12:52.494 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-07-06T20:12:52.494 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2023-07-06T20:12:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2023-07-06T20:12:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T20:12:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2023-07-06T20:12:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2023-07-06T20:12:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2023-07-06T20:12:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T20:12:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2023-07-06T20:12:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T20:12:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T20:12:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2023-07-06T20:12:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2023-07-06T20:12:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2023-07-06T20:12:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2023-07-06T20:12:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2023-07-06T20:12:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2023-07-06T20:12:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2023-07-06T20:12:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2023-07-06T20:12:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on skipped message = False 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = c2a9da8e-f30b-43f7-b235-3303ae838054 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2023-07-06T20:12:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2023-07-06T20:12:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2023-07-06T20:12:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2023-07-06T20:12:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2023-07-06T20:12:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: bdev async discard = True 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: bdev enable discard = True 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore allocator = bitmap 2023-07-06T20:12:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2023-07-06T20:12:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2023-07-06T20:12:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 1/20 2023-07-06T20:12:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 1/20 2023-07-06T20:12:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-07-06T20:12:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2023-07-06T20:12:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 4/10 2023-07-06T20:12:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T20:12:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2023-07-06T20:12:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2023-07-06T20:12:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2023-07-06T20:12:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2023-07-06T20:12:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-07-06T20:12:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.510 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2023-07-06T20:12:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-07-06T20:12:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2023-07-06T20:12:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-07-06T20:12:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-07-06T20:12:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2023-07-06T20:12:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T20:12:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2023-07-06T20:12:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: mon op complaint time = 120 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T20:12:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: client mount timeout = 600 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = "" 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2023-07-06T20:12:52.517 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds] 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds = 20 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds balancer = 20 2023-07-06T20:12:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug frag = True 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug scatterstat = True 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: mds op complaint time = 180 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: mds verify scatter = True 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-07-06T20:12:52.519 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-07-06T20:12:52.521 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-06T20:12:52.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-06T20:12:52.639 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-06T20:12:52.640 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:52.640 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-06T20:12:52.688 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:52.689 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-06T20:12:52.776 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:12:52.776 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:52.776 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T20:12:52.776 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T20:12:52.914 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:12:52.915 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:12:53.002 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:12:53.003 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:53.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T20:12:53.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T20:12:53.098 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:12:53.099 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:12:53.183 INFO:tasks.ceph:Setting up mon nodes... 2023-07-06T20:12:53.183 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-06T20:12:53.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T20:12:53.313 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T20:12:53.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T20:12:53.449 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T20:12:53.455 INFO:tasks.ceph:Setting up mds nodes... 2023-07-06T20:12:53.456 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T20:12:53.582 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T20:12:53.587 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-06T20:12:53.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T20:12:53.783 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T20:12:53.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-06T20:12:53.906 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T20:12:54.027 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T20:12:54.031 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-06T20:12:54.100 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T20:12:54.220 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T20:12:54.225 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-06T20:12:54.294 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-06T20:12:54.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-06T20:12:54.398 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-06T20:12:54.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-06T20:12:54.529 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-06T20:12:54.561 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-06T20:12:54.657 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-06T20:12:54.687 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-06T20:12:54.783 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-06T20:12:54.812 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-06T20:12:54.813 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T20:12:54.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-06T20:12:54.885 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T20:12:54.885 INFO:tasks.ceph:role: osd.0 2023-07-06T20:12:54.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:12:54.886 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T20:12:54.963 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:12:54.963 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:12:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:12:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:12:54.964 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:12:54.964 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:12:54.964 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:12:54.965 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:12:54.965 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:12:54.965 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:12:55.390 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-07-06T20:12:55.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-07-06T20:12:55.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-06T20:12:55.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-06T20:12:55.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-06T20:12:55.745 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T20:12:55.745 INFO:tasks.ceph:role: osd.1 2023-07-06T20:12:55.745 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:12:55.745 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T20:12:55.823 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:12:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:12:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:12:55.825 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:12:56.250 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-07-06T20:12:56.254 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-07-06T20:12:56.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-06T20:12:56.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-06T20:12:56.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-06T20:12:56.543 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T20:12:56.543 INFO:tasks.ceph:role: osd.2 2023-07-06T20:12:56.543 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:12:56.544 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T20:12:56.623 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:12:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:12:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:12:56.624 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:12:57.071 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-07-06T20:12:57.074 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-07-06T20:12:57.075 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-06T20:12:57.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-06T20:12:57.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-06T20:12:57.425 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T20:12:57.425 INFO:tasks.ceph:role: osd.3 2023-07-06T20:12:57.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:12:57.425 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T20:12:57.502 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:12:57.503 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:12:57.504 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:12:57.504 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:12:57.929 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-07-06T20:12:57.934 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-07-06T20:12:57.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-06T20:12:58.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-06T20:12:58.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:12:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:12:58.251+0000 7fc35d45b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-06T20:12:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:12:58.251+0000 7fc35d45b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-06T20:12:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:12:58.251+0000 7fc35d45b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T20:12:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:12:58.251+0000 7fc35d45b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T20:12:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:12:58.251+0000 7fc35d45b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T20:12:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:12:58.253+0000 7fc35d45b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-06T20:13:01.152 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-06T20:13:01.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:01.338+0000 7f8cffb1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-06T20:13:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:01.338+0000 7f8cffb1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-06T20:13:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:01.338+0000 7f8cffb1b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T20:13:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:01.339+0000 7f8cffb1b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T20:13:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:01.339+0000 7f8cffb1b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T20:13:01.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:01.340+0000 7f8cffb1b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-06T20:13:04.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-06T20:13:04.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:04.406+0000 7f77ee3cd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-06T20:13:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:04.406+0000 7f77ee3cd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-06T20:13:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:04.406+0000 7f77ee3cd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T20:13:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:04.406+0000 7f77ee3cd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T20:13:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:04.406+0000 7f77ee3cd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T20:13:04.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:04.408+0000 7f77ee3cd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-06T20:13:07.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-06T20:13:07.375 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:07.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:07.484+0000 7f0ba8d25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-06T20:13:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:07.485+0000 7f0ba8d25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-06T20:13:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:07.485+0000 7f0ba8d25640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T20:13:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:07.485+0000 7f0ba8d25640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T20:13:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:07.485+0000 7f0ba8d25640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T20:13:07.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:07.486+0000 7f0ba8d25640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-06T20:13:10.364 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-06T20:13:10.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-06T20:13:10.514 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T20:13:10.514 INFO:tasks.ceph:role: osd.4 2023-07-06T20:13:10.514 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:13:10.515 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T20:13:10.592 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:13:10.592 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:13:10.592 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:13:10.593 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:13:11.028 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-07-06T20:13:11.032 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-07-06T20:13:11.032 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-06T20:13:11.217 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-06T20:13:11.295 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-06T20:13:11.363 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T20:13:11.364 INFO:tasks.ceph:role: osd.5 2023-07-06T20:13:11.364 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:13:11.364 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T20:13:11.440 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:13:11.440 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:13:11.440 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:13:11.440 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:13:11.441 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:13:11.441 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:13:11.441 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:13:11.441 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:13:11.441 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:13:11.441 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:13:11.889 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-07-06T20:13:11.892 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-07-06T20:13:11.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-06T20:13:12.026 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-06T20:13:12.103 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-06T20:13:12.171 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T20:13:12.171 INFO:tasks.ceph:role: osd.6 2023-07-06T20:13:12.172 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:13:12.172 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T20:13:12.249 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:13:12.249 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:13:12.249 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:13:12.250 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:13:12.688 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-07-06T20:13:12.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-07-06T20:13:12.693 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-06T20:13:12.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-06T20:13:12.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-06T20:13:13.038 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T20:13:13.039 INFO:tasks.ceph:role: osd.7 2023-07-06T20:13:13.039 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:13:13.039 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T20:13:13.114 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T20:13:13.115 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T20:13:13.116 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T20:13:13.116 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T20:13:13.554 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-07-06T20:13:13.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-07-06T20:13:13.558 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-06T20:13:13.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-06T20:13:13.757 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:13.904+0000 7fb1d4c09640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-06T20:13:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:13.905+0000 7fb1d4c09640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-06T20:13:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:13.905+0000 7fb1d4c09640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T20:13:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:13.905+0000 7fb1d4c09640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T20:13:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:13.905+0000 7fb1d4c09640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T20:13:13.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:13.907+0000 7fb1d4c09640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-06T20:13:16.789 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-06T20:13:16.859 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:16.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:16.962+0000 7f8043d9d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-06T20:13:16.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:16.963+0000 7f8043d9d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-06T20:13:16.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:16.963+0000 7f8043d9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T20:13:16.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:16.963+0000 7f8043d9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T20:13:16.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:16.963+0000 7f8043d9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T20:13:16.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:16.965+0000 7f8043d9d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-06T20:13:19.846 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-06T20:13:19.915 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:20.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:13:20.017 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:13:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:20.019+0000 7fc4a0bfe640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-06T20:13:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:20.020+0000 7fc4a0bfe640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-06T20:13:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:20.020+0000 7fc4a0bfe640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T20:13:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:20.020+0000 7fc4a0bfe640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T20:13:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:20.020+0000 7fc4a0bfe640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T20:13:20.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:20.021+0000 7fc4a0bfe640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-06T20:13:22.910 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-06T20:13:22.980 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:23.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:23.082+0000 7f2af480c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-06T20:13:23.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:23.082+0000 7f2af480c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-06T20:13:23.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:23.083+0000 7f2af480c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T20:13:23.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:23.083+0000 7f2af480c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T20:13:23.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:23.083+0000 7f2af480c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T20:13:23.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:23.084+0000 7f2af480c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-06T20:13:25.964 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-06T20:13:26.034 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-06T20:13:26.035 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-06T20:13:26.112 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.112 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-06T20:13:26.184 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-06T20:13:26.254 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.254 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-06T20:13:26.364 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.364 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-06T20:13:26.473 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-06T20:13:26.582 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:26.583 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-06T20:13:26.692 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:26.692 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-06T20:13:26.766 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:26.766 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-06T20:13:26.835 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:26.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-06T20:13:26.943 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:26.943 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-06T20:13:27.051 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:27.052 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-06T20:13:27.159 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:27.159 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-06T20:13:27.266 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:27.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-06T20:13:27.374 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:27.375 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-06T20:13:27.423 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:27.424 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-06T20:13:27.470 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:27.470 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-06T20:13:27.515 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:27.516 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-06T20:13:27.600 INFO:tasks.ceph:Adding keys to all mons... 2023-07-06T20:13:27.600 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T20:13:27.605 DEBUG:teuthology.orchestra.run.smithi153:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T20:13:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.y] 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBFIKdkfs6mEhAARTSorZ0o/g9w2V6Dr7ArBg== 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBFIKdkt1qyIhAAV2Zt+4H6JHNHYPqFB0XaMQ== 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBFIKdkXnKqLhAAXTskx14TZ4a4QPMiZi40cw== 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2023-07-06T20:13:27.670 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBKIKdkDOsHDxAAk6iJhzgWm0nC2jkg5xqYPg== 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBNIKdk3p88FBAAxfAk8Bh/1X5hyaAsauDmDA== 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBQIKdk7yRFGBAAwSBhNBHWe6LAim+0MEhLqg== 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBTIKdkLkbzHBAAaehxTk2QxX+ezYStqt6jPQ== 2023-07-06T20:13:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBFIKdkigi+GhAAKT/PZgsjm5sUIw5t3MXX1w== 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkCnGUARAAvSNtHM2BJDXHrEB3lIgd2Q== 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkJQMdDRAA8qma6ry2Uk87MqanA8+wNA== 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBZIKdkEcr+NRAAkb0WVzKF+o/k3L3Hm8k/Tg== 2023-07-06T20:13:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBcIKdkU0Z0ORAA6RcJRYFnzuyoySt1Jws6Eg== 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBgIKdkI9M+ARAAiyzZQS7rS182eKUu9hNazA== 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBjIKdkDPf6BBAAEBZ8o1wlrgXFk0zD5oy0aQ== 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkSs+yFxAABbJ2vpRTPGJa6RY1/Ha9/Q== 2023-07-06T20:13:27.673 INFO:teuthology.orchestra.run.smithi027.stdout:[client.2] 2023-07-06T20:13:27.674 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkeD+IHxAAwuy2BoFocQRSfJX1NcVCZw== 2023-07-06T20:13:27.674 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2023-07-06T20:13:27.674 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkIZgrJxAALjLhrHI8h1rmxJdwrYjp0g== 2023-07-06T20:13:27.674 INFO:teuthology.orchestra.run.smithi027.stdout:[client.3] 2023-07-06T20:13:27.674 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkscKnLhAAdw7o+z02ZMNZ+MGucPmEsw== 2023-07-06T20:13:27.705 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.y] 2023-07-06T20:13:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBFIKdkfs6mEhAARTSorZ0o/g9w2V6Dr7ArBg== 2023-07-06T20:13:27.705 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-07-06T20:13:27.705 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBFIKdkt1qyIhAAV2Zt+4H6JHNHYPqFB0XaMQ== 2023-07-06T20:13:27.705 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBFIKdkXnKqLhAAXTskx14TZ4a4QPMiZi40cw== 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.0] 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBKIKdkDOsHDxAAk6iJhzgWm0nC2jkg5xqYPg== 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.1] 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBNIKdk3p88FBAAxfAk8Bh/1X5hyaAsauDmDA== 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.2] 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBQIKdk7yRFGBAAwSBhNBHWe6LAim+0MEhLqg== 2023-07-06T20:13:27.706 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.3] 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBTIKdkLkbzHBAAaehxTk2QxX+ezYStqt6jPQ== 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.x] 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBFIKdkigi+GhAAKT/PZgsjm5sUIw5t3MXX1w== 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBGIKdkCnGUARAAvSNtHM2BJDXHrEB3lIgd2Q== 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBGIKdkJQMdDRAA8qma6ry2Uk87MqanA8+wNA== 2023-07-06T20:13:27.707 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.4] 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBZIKdkEcr+NRAAkb0WVzKF+o/k3L3Hm8k/Tg== 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.5] 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBcIKdkU0Z0ORAA6RcJRYFnzuyoySt1Jws6Eg== 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.6] 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBgIKdkI9M+ARAAiyzZQS7rS182eKUu9hNazA== 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.7] 2023-07-06T20:13:27.708 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBjIKdkDPf6BBAAEBZ8o1wlrgXFk0zD5oy0aQ== 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBGIKdkSs+yFxAABbJ2vpRTPGJa6RY1/Ha9/Q== 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout:[client.2] 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBGIKdkeD+IHxAAwuy2BoFocQRSfJX1NcVCZw== 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout:[client.1] 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBGIKdkIZgrJxAALjLhrHI8h1rmxJdwrYjp0g== 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout:[client.3] 2023-07-06T20:13:27.709 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBGIKdkscKnLhAAdw7o+z02ZMNZ+MGucPmEsw== 2023-07-06T20:13:27.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T20:13:27.719 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T20:13:27.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:27.861 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:27.974 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:27.978 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:28.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.094 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.205 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.429 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T20:13:28.541 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T20:13:28.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:28.653 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:28.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:28.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T20:13:28.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.885 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:28.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:29.106 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:29.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:29.219 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:29.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T20:13:29.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.336 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.451 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.567 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.682 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T20:13:29.794 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-06T20:13:29.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-06T20:13:29.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T20:13:29.991 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-06T20:13:30.062 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-06T20:13:30.132 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T20:13:30.253 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-06T20:13:30.322 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-06T20:13:30.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T20:13:30.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-06T20:13:30.579 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:30.584 DEBUG:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T20:13:30.628 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-06T20:13:30.628 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-06T20:13:30.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-06T20:13:30.633 INFO:tasks.ceph.mon.a:Started 2023-07-06T20:13:30.633 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-06T20:13:30.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-06T20:13:30.672 INFO:tasks.ceph.mon.b:Started 2023-07-06T20:13:30.673 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-06T20:13:30.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-06T20:13:30.677 INFO:tasks.ceph.mon.c:Started 2023-07-06T20:13:30.677 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-06T20:13:30.678 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-06T20:13:30.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-06T20:13:30.682 INFO:tasks.ceph.mgr.y:Started 2023-07-06T20:13:30.682 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-06T20:13:30.682 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-06T20:13:30.688 INFO:tasks.ceph.mgr.x:Started 2023-07-06T20:13:30.688 INFO:tasks.ceph:Setting crush tunables to default 2023-07-06T20:13:30.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2023-07-06T20:13:30.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.889+0000 7f56251ff700 1 Processor -- start 2023-07-06T20:13:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.889+0000 7f56251ff700 1 -- start start 2023-07-06T20:13:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f56251ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 0x7f56200d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f56251ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f56200d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f56251ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56200cf780 con 0x7f56200b60e0 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f56251ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56200cd490 con 0x7f562015fca0 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f56251ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cd770 con 0x7f56200b5c90 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f561e59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f56200d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f561e59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f56200d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:40016/0 (socket says 172.21.15.27:40016) 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f561e59c700 1 -- 172.21.15.27:0/3245798311 learned_addr learned my addr 172.21.15.27:0/3245798311 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.890+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 0x7f56200d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561e59c700 1 -- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 msgr2=0x7f56200d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561e59c700 1 -- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 msgr2=0x7f56200d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561e59c700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f56200d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561e59c700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f56200d2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561ed9d700 1 -- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 msgr2=0x7f56200d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561ed9d700 1 -- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 msgr2=0x7f56200d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T20:13:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 0x7f56200d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T20:13:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:30.891+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 0x7f56200d1420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:13:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.040+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210269298 0 0) 0x7f56200cf780 con 0x7f56200b60e0 2023-07-06T20:13:31.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56080035b0 con 0x7f56200b60e0 2023-07-06T20:13:31.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3660347091 0 0) 0x7f56080035b0 con 0x7f56200b60e0 2023-07-06T20:13:31.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 msgr2=0x7f56200d2d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f56200d2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 msgr2=0x7f56200d1420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 --2- 172.21.15.27:0/3245798311 >> v2:172.21.15.153:3300/0 conn(0x7f562015fca0 0x7f56200d1420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200cda50 con 0x7f56200b60e0 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f56180019f0 con 0x7f56200b60e0 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 >> v1:172.21.15.153:6789/0 conn(0x7f56200b60e0 legacy=0x7f56200cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5608003a00 0x7f5608003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608004850 0x7f5608004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f5608005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cd490 con 0x7f5608003a00 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cd770 con 0x7f5608004850 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.041+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560800bb40 con 0x7f56200b5c90 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561e59c700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608004850 0x7f5608004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5608003a00 0x7f5608003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:31.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561f59e700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f5608005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561ed9d700 1 -- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 msgr2=0x7f5608005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f5608005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561ed9d700 1 -- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608004850 msgr2=0x7f5608004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608004850 0x7f5608004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561ed9d700 1 -- 172.21.15.27:0/3245798311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080035b0 con 0x7f5608003a00 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.042+0000 7f561f59e700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56200b5c90 0x7f5608005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:13:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.043+0000 7f561ed9d700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5608003a00 0x7f5608003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5610009890 tx=0x7f5608005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01c1fcecc6ae0ac server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.044+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000c070 con 0x7f5608003a00 2023-07-06T20:13:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610004020 con 0x7f5608003a00 2023-07-06T20:13:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f561dd9b700 1 -- 172.21.15.27:0/3245798311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56100043b0 con 0x7f5608003a00 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f56251ff700 1 -- 172.21.15.27:0/3245798311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5608003a00 msgr2=0x7f5608003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f56251ff700 1 --2- 172.21.15.27:0/3245798311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5608003a00 0x7f5608003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5610009890 tx=0x7f5608005880 comp rx=0 tx=0).stop 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f561f59e700 1 -- 172.21.15.27:0/3245798311 reap_dead start 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f56251ff700 1 -- 172.21.15.27:0/3245798311 shutdown_connections 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.045+0000 7f56251ff700 1 -- 172.21.15.27:0/3245798311 >> 172.21.15.27:0/3245798311 conn(0x7f562006c150 msgr2=0x7f562006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.046+0000 7f56251ff700 1 -- 172.21.15.27:0/3245798311 shutdown_connections 2023-07-06T20:13:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.046+0000 7f56251ff700 1 -- 172.21.15.27:0/3245798311 wait complete. 2023-07-06T20:13:31.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.048+0000 7f56251ff700 1 Processor -- start 2023-07-06T20:13:31.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.048+0000 7f56251ff700 1 -- start start 2023-07-06T20:13:31.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f56251ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 0x7f562014fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:31.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f56251ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56201f6380 0x7f562015ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:31.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f56251ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56201593d0 0x7f5620159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:31.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f56251ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56201513e0 con 0x7f56201f6380 2023-07-06T20:13:31.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f56251ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56201f68d0 con 0x7f5608003a00 2023-07-06T20:13:31.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f56251ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5620159d90 con 0x7f56201593d0 2023-07-06T20:13:31.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.049+0000 7f561ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 0x7f562014fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 0x7f562014fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:44466/0 (socket says 172.21.15.27:44466) 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 -- 172.21.15.27:0/540287824 learned_addr learned my addr 172.21.15.27:0/540287824 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561e59c700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56201f6380 0x7f562015ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561f59e700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56201593d0 0x7f5620159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 -- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56201593d0 msgr2=0x7f5620159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56201593d0 0x7f5620159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 -- 172.21.15.27:0/540287824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56201f6380 msgr2=0x7f562015ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:31.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56201f6380 0x7f562015ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 -- 172.21.15.27:0/540287824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562015f4d0 con 0x7f5608003a00 2023-07-06T20:13:31.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561e59c700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56201f6380 0x7f562015ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:31.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.050+0000 7f561ed9d700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 0x7f562014fae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56100038d0 tx=0x7f56100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.051+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56100045a0 con 0x7f5608003a00 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.051+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610003d00 con 0x7f5608003a00 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.051+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56200cbd90 con 0x7f5608003a00 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.051+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561001d6a0 con 0x7f5608003a00 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.051+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56200cc350 con 0x7f5608003a00 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.052+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f561001db00 con 0x7f5608003a00 2023-07-06T20:13:31.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.052+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f561002fbd0 con 0x7f5608003a00 2023-07-06T20:13:31.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.053+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5620151520 con 0x7f5608003a00 2023-07-06T20:13:31.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.056+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5620151520 con 0x7f5608003a00 2023-07-06T20:13:31.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.135+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f56200cf050 con 0x7f5608003a00 2023-07-06T20:13:31.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.139+0000 7f56077fe700 1 -- 172.21.15.27:0/540287824 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f561002a070 con 0x7f5608003a00 2023-07-06T20:13:31.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 msgr2=0x7f562014fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:31.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 0x7f562014fae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56100038d0 tx=0x7f56100047c0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 shutdown_connections 2023-07-06T20:13:31.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f56201593d0 0x7f5620159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5608003a00 0x7f562014fae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 --2- 172.21.15.27:0/540287824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56201f6380 0x7f562015ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 >> 172.21.15.27:0/540287824 conn(0x7f562006c150 msgr2=0x7f56200cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.140+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 shutdown_connections 2023-07-06T20:13:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:31.141+0000 7f56251ff700 1 -- 172.21.15.27:0/540287824 wait complete. 2023-07-06T20:13:31.143 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2023-07-06T20:13:31.159 INFO:tasks.ceph:check_enable_crimson: False 2023-07-06T20:13:31.159 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-06T20:13:31.160 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:31.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-06T20:13:31.213 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:31.212+0000 7fdb245b3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:13:31.218 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:31.216+0000 7fcb0ea3b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:13:31.230 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:31.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-06T20:13:31.341 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:31.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-06T20:13:31.433 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:31.432+0000 7fcb0ea3b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:13:31.460 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:31.458+0000 7fdb245b3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:13:31.469 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:13:31.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-06T20:13:31.540 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:31.541 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-06T20:13:31.610 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:31.610 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-06T20:13:31.719 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:31.719 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-06T20:13:31.835 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:13:31.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-06T20:13:31.886 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:31.885+0000 7fcb0ea3b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:13:31.903 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new cfbccfdc-2546-4e99-b098-57a4455400fe 0 2023-07-06T20:13:31.914 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:31.912+0000 7fdb245b3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:13:32.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.134+0000 7f2bd05ff700 1 Processor -- start 2023-07-06T20:13:32.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.134+0000 7f2bd05ff700 1 -- start start 2023-07-06T20:13:32.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.135+0000 7f2bd05ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc80b6110 0x7f2bc80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.135+0000 7f2bd05ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80cf780 con 0x7f2bc80b5c90 2023-07-06T20:13:32.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bd05ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80cd490 con 0x7f2bc815fca0 2023-07-06T20:13:32.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bd05ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80cd770 con 0x7f2bc80b6110 2023-07-06T20:13:32.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bceb9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc80b6110 0x7f2bc80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bce39b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2bc815fca0 0x7f2bc80d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:43484/0 (socket says 172.21.15.153:43484) 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bce39b700 1 -- 172.21.15.153:0/1990015442 learned_addr learned my addr 172.21.15.153:0/1990015442 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bceb9c700 1 -- 172.21.15.153:0/1990015442 >> v1:172.21.15.153:6789/0 conn(0x7f2bc815fca0 legacy=0x7f2bc80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bcd399700 1 -- 172.21.15.153:0/1990015442 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919435636 0 0) 0x7f2bc80cd490 con 0x7f2bc815fca0 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bceb9c700 1 -- 172.21.15.153:0/1990015442 >> v1:172.21.15.27:6789/0 conn(0x7f2bc80b5c90 legacy=0x7f2bc80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bceb9c700 1 -- 172.21.15.153:0/1990015442 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80cda50 con 0x7f2bc80b6110 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.136+0000 7f2bceb9c700 1 --2- 172.21.15.153:0/1990015442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc80b6110 0x7f2bc80cfc10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009dc0 tx=0x7f2bc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e19a4e672e21ce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:32.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bcd399700 1 -- 172.21.15.153:0/1990015442 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000e070 con 0x7f2bc80b6110 2023-07-06T20:13:32.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bcd399700 1 -- 172.21.15.153:0/1990015442 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000bad0 con 0x7f2bc80b6110 2023-07-06T20:13:32.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bcd399700 1 -- 172.21.15.153:0/1990015442 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000be80 con 0x7f2bc80b6110 2023-07-06T20:13:32.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bd05ff700 1 -- 172.21.15.153:0/1990015442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc80b6110 msgr2=0x7f2bc80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bd05ff700 1 --2- 172.21.15.153:0/1990015442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc80b6110 0x7f2bc80cfc10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009dc0 tx=0x7f2bc000b330 comp rx=0 tx=0).stop 2023-07-06T20:13:32.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bd05ff700 1 -- 172.21.15.153:0/1990015442 shutdown_connections 2023-07-06T20:13:32.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bd05ff700 1 --2- 172.21.15.153:0/1990015442 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc80b6110 0x7f2bc80cfc10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bd05ff700 1 -- 172.21.15.153:0/1990015442 >> 172.21.15.153:0/1990015442 conn(0x7f2bc806c150 msgr2=0x7f2bc806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:32.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.137+0000 7f2bd05ff700 1 -- 172.21.15.153:0/1990015442 shutdown_connections 2023-07-06T20:13:32.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.138+0000 7f2bd05ff700 1 -- 172.21.15.153:0/1990015442 wait complete. 2023-07-06T20:13:32.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.139+0000 7f2bd05ff700 1 Processor -- start 2023-07-06T20:13:32.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.139+0000 7f2bd05ff700 1 -- start start 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bd05ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bc80b5c90 0x7f2bc815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bd05ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc80b6110 0x7f2bc815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bd05ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc815fca0 0x7f2bc81f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bd05ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc815ea50 con 0x7f2bc80b6110 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bd05ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc815e1d0 con 0x7f2bc80b5c90 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bd05ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc815e490 con 0x7f2bc815fca0 2023-07-06T20:13:32.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bce39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bc80b5c90 0x7f2bc815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bceb9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc815fca0 0x7f2bc81f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.140+0000 7f2bce39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bc80b5c90 0x7f2bc815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33126/0 (socket says 172.21.15.153:33126) 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bce39b700 1 -- 172.21.15.153:0/3536963437 learned_addr learned my addr 172.21.15.153:0/3536963437 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bcdb9a700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc80b6110 0x7f2bc815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bceb9c700 1 -- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bc80b5c90 msgr2=0x7f2bc815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bceb9c700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bc80b5c90 0x7f2bc815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bceb9c700 1 -- 172.21.15.153:0/3536963437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc80b6110 msgr2=0x7f2bc815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bceb9c700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc80b6110 0x7f2bc815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bceb9c700 1 -- 172.21.15.153:0/3536963437 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009a70 con 0x7f2bc815fca0 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bcdb9a700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc80b6110 0x7f2bc815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bceb9c700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc815fca0 0x7f2bc81f1ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0015000 tx=0x7f2bc0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0012630 con 0x7f2bc815fca0 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00127b0 con 0x7f2bc815fca0 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.141+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0012b60 con 0x7f2bc815fca0 2023-07-06T20:13:32.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.142+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc815e770 con 0x7f2bc815fca0 2023-07-06T20:13:32.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.142+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc815cc20 con 0x7f2bc815fca0 2023-07-06T20:13:32.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.142+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2bc001d420 con 0x7f2bc815fca0 2023-07-06T20:13:32.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.142+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2bc00257d0 con 0x7f2bc815fca0 2023-07-06T20:13:32.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.143+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc80008d0 con 0x7f2bc815fca0 2023-07-06T20:13:32.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.146+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2bc002a060 con 0x7f2bc815fca0 2023-07-06T20:13:32.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.233+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cfbccfdc-2546-4e99-b098-57a4455400fe", "id": 0} v 0) v1 -- 0x7f2bc8066010 con 0x7f2bc815fca0 2023-07-06T20:13:32.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.241+0000 7f2bbeffd700 1 -- 172.21.15.153:0/3536963437 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfbccfdc-2546-4e99-b098-57a4455400fe", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2bc0025a60 con 0x7f2bc815fca0 2023-07-06T20:13:32.243 INFO:teuthology.orchestra.run.smithi153.stdout:0 2023-07-06T20:13:32.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc815fca0 msgr2=0x7f2bc81f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc815fca0 0x7f2bc81f1ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0015000 tx=0x7f2bc0004ad0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 shutdown_connections 2023-07-06T20:13:32.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2bc815fca0 0x7f2bc81f1ec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bc80b5c90 0x7f2bc815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 --2- 172.21.15.153:0/3536963437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bc80b6110 0x7f2bc815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 >> 172.21.15.153:0/3536963437 conn(0x7f2bc806c150 msgr2=0x7f2bc80d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:32.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 shutdown_connections 2023-07-06T20:13:32.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.242+0000 7f2bd05ff700 1 -- 172.21.15.153:0/3536963437 wait complete. 2023-07-06T20:13:32.261 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 42558411-4aac-4fbd-8e61-4349de6d8b6a 1 2023-07-06T20:13:32.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.456+0000 7fbc6abff700 1 Processor -- start 2023-07-06T20:13:32.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.456+0000 7fbc6abff700 1 -- start start 2023-07-06T20:13:32.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.457+0000 7fbc6abff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc64159fc0 0x7fbc640d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.457+0000 7fbc6abff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc640cf0a0 0x7fbc6415d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.457+0000 7fbc6abff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640cf540 con 0x7fbc6415a390 2023-07-06T20:13:32.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.457+0000 7fbc6abff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc640cf820 con 0x7fbc64159fc0 2023-07-06T20:13:32.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.457+0000 7fbc6abff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640cdcb0 con 0x7fbc640cf0a0 2023-07-06T20:13:32.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc640cf0a0 0x7fbc6415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc6919c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbc6415a390 0x7fbc640b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:43488/0 (socket says 172.21.15.153:43488) 2023-07-06T20:13:32.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc6919c700 1 -- 172.21.15.153:0/4143019406 learned_addr learned my addr 172.21.15.153:0/4143019406 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc6899b700 1 --2- 172.21.15.153:0/4143019406 >> v2:172.21.15.27:3300/0 conn(0x7fbc64159fc0 0x7fbc640d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc637fe700 1 -- 172.21.15.153:0/4143019406 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403856690 0 0) 0x7fbc640cf540 con 0x7fbc6415a390 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc637fe700 1 -- 172.21.15.153:0/4143019406 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc540035b0 con 0x7fbc6415a390 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc63fff700 1 -- 172.21.15.153:0/4143019406 >> v2:172.21.15.27:3300/0 conn(0x7fbc64159fc0 msgr2=0x7fbc640d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc63fff700 1 --2- 172.21.15.153:0/4143019406 >> v2:172.21.15.27:3300/0 conn(0x7fbc64159fc0 0x7fbc640d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc63fff700 1 -- 172.21.15.153:0/4143019406 >> v1:172.21.15.153:6789/0 conn(0x7fbc6415a390 legacy=0x7fbc640b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc63fff700 1 -- 172.21.15.153:0/4143019406 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640cd660 con 0x7fbc640cf0a0 2023-07-06T20:13:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.458+0000 7fbc6899b700 1 --2- 172.21.15.153:0/4143019406 >> v2:172.21.15.27:3300/0 conn(0x7fbc64159fc0 0x7fbc640d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.459+0000 7fbc63fff700 1 --2- 172.21.15.153:0/4143019406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc640cf0a0 0x7fbc6415d680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009840 tx=0x7fbc50009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d94987416fa1c5c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.459+0000 7fbc637fe700 1 -- 172.21.15.153:0/4143019406 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50011830 con 0x7fbc640cf0a0 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.459+0000 7fbc637fe700 1 -- 172.21.15.153:0/4143019406 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc50011990 con 0x7fbc640cf0a0 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.459+0000 7fbc637fe700 1 -- 172.21.15.153:0/4143019406 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50011d20 con 0x7fbc640cf0a0 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.459+0000 7fbc6abff700 1 -- 172.21.15.153:0/4143019406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc640cf0a0 msgr2=0x7fbc6415d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.459+0000 7fbc6abff700 1 --2- 172.21.15.153:0/4143019406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc640cf0a0 0x7fbc6415d680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009840 tx=0x7fbc50009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.460+0000 7fbc6abff700 1 -- 172.21.15.153:0/4143019406 shutdown_connections 2023-07-06T20:13:32.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.460+0000 7fbc6abff700 1 --2- 172.21.15.153:0/4143019406 >> v2:172.21.15.27:3300/0 conn(0x7fbc64159fc0 0x7fbc640d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.460+0000 7fbc6abff700 1 --2- 172.21.15.153:0/4143019406 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc640cf0a0 0x7fbc6415d680 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.460+0000 7fbc6abff700 1 -- 172.21.15.153:0/4143019406 >> 172.21.15.153:0/4143019406 conn(0x7fbc6406c1e0 msgr2=0x7fbc6406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.460+0000 7fbc6abff700 1 -- 172.21.15.153:0/4143019406 shutdown_connections 2023-07-06T20:13:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.460+0000 7fbc6abff700 1 -- 172.21.15.153:0/4143019406 wait complete. 2023-07-06T20:13:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.461+0000 7fbc6abff700 1 Processor -- start 2023-07-06T20:13:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 -- start start 2023-07-06T20:13:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 0x7fbc641f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc64159fc0 0x7fbc641587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc6415a390 0x7fbc64158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640c6740 con 0x7fbc64159fc0 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc641f2340 con 0x7fbc640cf0a0 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.462+0000 7fbc6abff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64153d40 con 0x7fbc6415a390 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 0x7fbc641f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6919c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc6415a390 0x7fbc64158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc64159fc0 0x7fbc641587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 0x7fbc641f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33138/0 (socket says 172.21.15.153:33138) 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 -- 172.21.15.153:0/2332301678 learned_addr learned my addr 172.21.15.153:0/2332301678 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 -- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc6415a390 msgr2=0x7fbc64158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc6415a390 0x7fbc64158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 -- 172.21.15.153:0/2332301678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc64159fc0 msgr2=0x7fbc641587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc64159fc0 0x7fbc641587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6899b700 1 -- 172.21.15.153:0/2332301678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580094a0 con 0x7fbc640cf0a0 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc63fff700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc64159fc0 0x7fbc641587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.463+0000 7fbc6919c700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc6415a390 0x7fbc64158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:32.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc6899b700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 0x7fbc641f1e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800ebe0 tx=0x7fbc5800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800c850 con 0x7fbc640cf0a0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5800c9b0 con 0x7fbc640cf0a0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800cd40 con 0x7fbc640cf0a0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc500094a0 con 0x7fbc640cf0a0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc64154300 con 0x7fbc640cf0a0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbc58005d20 con 0x7fbc640cf0a0 2023-07-06T20:13:32.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.464+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fbc58013820 con 0x7fbc640cf0a0 2023-07-06T20:13:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.465+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64068190 con 0x7fbc640cf0a0 2023-07-06T20:13:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.468+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbc58021060 con 0x7fbc640cf0a0 2023-07-06T20:13:32.518 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:32.516+0000 7fcb0ea3b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:13:32.538 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:32.536+0000 7fdb245b3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:13:32.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.554+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "42558411-4aac-4fbd-8e61-4349de6d8b6a", "id": 1} v 0) v1 -- 0x7fbc640cc9a0 con 0x7fbc640cf0a0 2023-07-06T20:13:32.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.562+0000 7fbc617fa700 1 -- 172.21.15.153:0/2332301678 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42558411-4aac-4fbd-8e61-4349de6d8b6a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc58029020 con 0x7fbc640cf0a0 2023-07-06T20:13:32.563 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-07-06T20:13:32.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 msgr2=0x7fbc641f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 0x7fbc641f1e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800ebe0 tx=0x7fbc5800eef0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 shutdown_connections 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc6415a390 0x7fbc64158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc640cf0a0 0x7fbc641f1e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 --2- 172.21.15.153:0/2332301678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc64159fc0 0x7fbc641587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 >> 172.21.15.153:0/2332301678 conn(0x7fbc6406c1e0 msgr2=0x7fbc640d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 shutdown_connections 2023-07-06T20:13:32.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.563+0000 7fbc6abff700 1 -- 172.21.15.153:0/2332301678 wait complete. 2023-07-06T20:13:32.582 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new dee0e9d7-ef17-4054-a52f-2ff3bc8534e0 2 2023-07-06T20:13:32.772 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:32.770+0000 7fcb0ea3b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:13:32.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.775+0000 7f0cf9dff700 1 Processor -- start 2023-07-06T20:13:32.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.775+0000 7f0cf9dff700 1 -- start start 2023-07-06T20:13:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.776+0000 7f0cf9dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf40b6110 0x7f0cf40cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.776+0000 7f0cf9dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40cf780 con 0x7f0cf40b5c90 2023-07-06T20:13:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.776+0000 7f0cf9dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40cd490 con 0x7f0cf415fca0 2023-07-06T20:13:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.776+0000 7f0cf9dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40cd770 con 0x7f0cf40b6110 2023-07-06T20:13:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf37fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0cf415fca0 0x7f0cf40d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:43492/0 (socket says 172.21.15.153:43492) 2023-07-06T20:13:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf37fe700 1 -- 172.21.15.153:0/1107624808 learned_addr learned my addr 172.21.15.153:0/1107624808 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-07-06T20:13:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf3fff700 1 --2- 172.21.15.153:0/1107624808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf40b6110 0x7f0cf40cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf3fff700 1 -- 172.21.15.153:0/1107624808 >> v1:172.21.15.153:6789/0 conn(0x7f0cf415fca0 legacy=0x7f0cf40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf3fff700 1 -- 172.21.15.153:0/1107624808 >> v1:172.21.15.27:6789/0 conn(0x7f0cf40b5c90 legacy=0x7f0cf40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf3fff700 1 -- 172.21.15.153:0/1107624808 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf40cda50 con 0x7f0cf40b6110 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf27fc700 1 -- 172.21.15.153:0/1107624808 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127394831 0 0) 0x7f0cf40cd490 con 0x7f0cf415fca0 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.777+0000 7f0cf3fff700 1 --2- 172.21.15.153:0/1107624808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf40b6110 0x7f0cf40cfc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009dc0 tx=0x7f0cec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1214b5de530dc204 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf27fc700 1 -- 172.21.15.153:0/1107624808 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00e070 con 0x7f0cf40b6110 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf27fc700 1 -- 172.21.15.153:0/1107624808 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec00bad0 con 0x7f0cf40b6110 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf27fc700 1 -- 172.21.15.153:0/1107624808 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00be80 con 0x7f0cf40b6110 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf9dff700 1 -- 172.21.15.153:0/1107624808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf40b6110 msgr2=0x7f0cf40cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf9dff700 1 --2- 172.21.15.153:0/1107624808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf40b6110 0x7f0cf40cfc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009dc0 tx=0x7f0cec00b330 comp rx=0 tx=0).stop 2023-07-06T20:13:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf9dff700 1 -- 172.21.15.153:0/1107624808 shutdown_connections 2023-07-06T20:13:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf9dff700 1 --2- 172.21.15.153:0/1107624808 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf40b6110 0x7f0cf40cfc10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf9dff700 1 -- 172.21.15.153:0/1107624808 >> 172.21.15.153:0/1107624808 conn(0x7f0cf406c150 msgr2=0x7f0cf406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.778+0000 7f0cf9dff700 1 -- 172.21.15.153:0/1107624808 shutdown_connections 2023-07-06T20:13:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.779+0000 7f0cf9dff700 1 -- 172.21.15.153:0/1107624808 wait complete. 2023-07-06T20:13:32.783 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:32.781+0000 7fdb245b3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:13:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.780+0000 7f0cf9dff700 1 Processor -- start 2023-07-06T20:13:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.780+0000 7f0cf9dff700 1 -- start start 2023-07-06T20:13:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf9dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 0x7f0cf415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf9dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf40b6110 0x7f0cf40c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf9dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf415fca0 0x7f0cf4146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf9dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4158920 con 0x7f0cf40b6110 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf9dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf415d120 con 0x7f0cf40b5c90 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf9dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41450b0 con 0x7f0cf415fca0 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.781+0000 7f0cf37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 0x7f0cf415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 0x7f0cf415cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33150/0 (socket says 172.21.15.153:33150) 2023-07-06T20:13:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 -- 172.21.15.153:0/964311722 learned_addr learned my addr 172.21.15.153:0/964311722 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf2ffd700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf40b6110 0x7f0cf40c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf3fff700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf415fca0 0x7f0cf4146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 -- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf415fca0 msgr2=0x7f0cf4146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf415fca0 0x7f0cf4146150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 -- 172.21.15.153:0/964311722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf40b6110 msgr2=0x7f0cf40c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf40b6110 0x7f0cf40c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 -- 172.21.15.153:0/964311722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec009a70 con 0x7f0cf40b5c90 2023-07-06T20:13:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cf37fe700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 0x7f0cf415cbe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c4d0 tx=0x7f0ce400c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:32.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.782+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004c80 con 0x7f0cf40b5c90 2023-07-06T20:13:32.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.783+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce4004de0 con 0x7f0cf40b5c90 2023-07-06T20:13:32.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.783+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4007310 con 0x7f0cf40b5c90 2023-07-06T20:13:32.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.783+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf4145390 con 0x7f0cf40b5c90 2023-07-06T20:13:32.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.783+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4145950 con 0x7f0cf40b5c90 2023-07-06T20:13:32.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.783+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0ce401d3f0 con 0x7f0cf40b5c90 2023-07-06T20:13:32.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.783+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0ce401dac0 con 0x7f0cf40b5c90 2023-07-06T20:13:32.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.784+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf40008d0 con 0x7f0cf40b5c90 2023-07-06T20:13:32.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.786+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f0ce401dda0 con 0x7f0cf40b5c90 2023-07-06T20:13:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.873+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dee0e9d7-ef17-4054-a52f-2ff3bc8534e0", "id": 2} v 0) v1 -- 0x7f0cf40b2b10 con 0x7f0cf40b5c90 2023-07-06T20:13:32.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.880+0000 7f0cdbfff700 1 -- 172.21.15.153:0/964311722 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dee0e9d7-ef17-4054-a52f-2ff3bc8534e0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ce401d6a0 con 0x7f0cf40b5c90 2023-07-06T20:13:32.882 INFO:teuthology.orchestra.run.smithi153.stdout:2 2023-07-06T20:13:32.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 msgr2=0x7f0cf415cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:32.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 0x7f0cf415cbe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c4d0 tx=0x7f0ce400c7e0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 shutdown_connections 2023-07-06T20:13:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0cf415fca0 0x7f0cf4146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf40b5c90 0x7f0cf415cbe0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 --2- 172.21.15.153:0/964311722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cf40b6110 0x7f0cf40c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.881+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 >> 172.21.15.153:0/964311722 conn(0x7f0cf406c150 msgr2=0x7f0cf40d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.882+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 shutdown_connections 2023-07-06T20:13:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:32.882+0000 7f0cf9dff700 1 -- 172.21.15.153:0/964311722 wait complete. 2023-07-06T20:13:32.900 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 86e15cd3-5bb1-4ce2-ad04-f0235fd0782e 3 2023-07-06T20:13:33.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.093+0000 7f561afff700 1 Processor -- start 2023-07-06T20:13:33.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.093+0000 7f561afff700 1 -- start start 2023-07-06T20:13:33.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.094+0000 7f561afff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f56140b5c90 0x7f56140d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f561afff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56140cf780 con 0x7f561415fca0 2023-07-06T20:13:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f561afff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56140cd490 con 0x7f56140b6110 2023-07-06T20:13:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f561afff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56140cd770 con 0x7f56140b5c90 2023-07-06T20:13:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f561959c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f56140b6110 0x7f56140cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:43506/0 (socket says 172.21.15.153:43506) 2023-07-06T20:13:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f5613fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f56140b5c90 0x7f56140d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f561959c700 1 -- 172.21.15.153:0/2079561634 learned_addr learned my addr 172.21.15.153:0/2079561634 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-07-06T20:13:33.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190846658 0 0) 0x7f56140cf780 con 0x7f561415fca0 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56040035b0 con 0x7f561415fca0 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589539420 0 0) 0x7f56140cd490 con 0x7f56140b6110 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56140cf780 con 0x7f56140b6110 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f5613fff700 1 -- 172.21.15.153:0/2079561634 >> v1:172.21.15.153:6789/0 conn(0x7f56140b6110 legacy=0x7f56140cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f5613fff700 1 -- 172.21.15.153:0/2079561634 >> v1:172.21.15.27:6789/0 conn(0x7f561415fca0 legacy=0x7f56140d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.095+0000 7f5613fff700 1 -- 172.21.15.153:0/2079561634 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140cda50 con 0x7f56140b5c90 2023-07-06T20:13:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.096+0000 7f5613fff700 1 --2- 172.21.15.153:0/2079561634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f56140d2d40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5600009840 tx=0x7f5600009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d98b528d2d90aec server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.096+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600011830 con 0x7f56140b5c90 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.096+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600011990 con 0x7f56140b5c90 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.096+0000 7f56137fe700 1 -- 172.21.15.153:0/2079561634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600011d20 con 0x7f56140b5c90 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 -- 172.21.15.153:0/2079561634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 msgr2=0x7f56140d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 --2- 172.21.15.153:0/2079561634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f56140d2d40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5600009840 tx=0x7f5600009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 -- 172.21.15.153:0/2079561634 shutdown_connections 2023-07-06T20:13:33.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 --2- 172.21.15.153:0/2079561634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f56140d2d40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 -- 172.21.15.153:0/2079561634 >> 172.21.15.153:0/2079561634 conn(0x7f561406c150 msgr2=0x7f561406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 -- 172.21.15.153:0/2079561634 shutdown_connections 2023-07-06T20:13:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.097+0000 7f561afff700 1 -- 172.21.15.153:0/2079561634 wait complete. 2023-07-06T20:13:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.100+0000 7f561afff700 1 Processor -- start 2023-07-06T20:13:33.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.100+0000 7f561afff700 1 -- start start 2023-07-06T20:13:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.101+0000 7f561afff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f561415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.101+0000 7f561afff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 0x7f56140c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f561afff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f561415fca0 0x7f561415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f561afff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56141f30d0 con 0x7f56140b5c90 2023-07-06T20:13:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f561afff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561415c780 con 0x7f56140b6110 2023-07-06T20:13:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f561afff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561415b9d0 con 0x7f561415fca0 2023-07-06T20:13:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5618d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f561415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5613fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 0x7f56140c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5613fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 0x7f56140c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33152/0 (socket says 172.21.15.153:33152) 2023-07-06T20:13:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5613fff700 1 -- 172.21.15.153:0/3211461947 learned_addr learned my addr 172.21.15.153:0/3211461947 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f561959c700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f561415fca0 0x7f561415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5618d9b700 1 -- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f561415fca0 msgr2=0x7f561415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5618d9b700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f561415fca0 0x7f561415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5618d9b700 1 -- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 msgr2=0x7f56140c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5618d9b700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 0x7f56140c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5618d9b700 1 -- 172.21.15.153:0/3211461947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000094a0 con 0x7f56140b5c90 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.102+0000 7f5613fff700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 0x7f56140c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f5618d9b700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f561415c240 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f560800c400 tx=0x7f560800c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080052d0 con 0x7f56140b5c90 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608005430 con 0x7f56140b5c90 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56141f2b30 con 0x7f56140b5c90 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080038c0 con 0x7f56140b5c90 2023-07-06T20:13:33.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561415bc70 con 0x7f56140b5c90 2023-07-06T20:13:33.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5608003d20 con 0x7f56140b5c90 2023-07-06T20:13:33.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.103+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f56080128b0 con 0x7f56140b5c90 2023-07-06T20:13:33.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.104+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561415b830 con 0x7f56140b5c90 2023-07-06T20:13:33.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.107+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f561415b830 con 0x7f56140b5c90 2023-07-06T20:13:33.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.193+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd new", "uuid": "86e15cd3-5bb1-4ce2-ad04-f0235fd0782e", "id": 3} v 0) v1 -- 0x7f56140008d0 con 0x7f56140b5c90 2023-07-06T20:13:33.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.198+0000 7f56117fa700 1 -- 172.21.15.153:0/3211461947 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "86e15cd3-5bb1-4ce2-ad04-f0235fd0782e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5608021060 con 0x7f56140b5c90 2023-07-06T20:13:33.200 INFO:teuthology.orchestra.run.smithi153.stdout:3 2023-07-06T20:13:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 msgr2=0x7f561415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f561415c240 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f560800c400 tx=0x7f560800c710 comp rx=0 tx=0).stop 2023-07-06T20:13:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 shutdown_connections 2023-07-06T20:13:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f561415fca0 0x7f561415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56140b5c90 0x7f561415c240 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 --2- 172.21.15.153:0/3211461947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56140b6110 0x7f56140c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 >> 172.21.15.153:0/3211461947 conn(0x7f561406c150 msgr2=0x7f56140cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 shutdown_connections 2023-07-06T20:13:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.199+0000 7f561afff700 1 -- 172.21.15.153:0/3211461947 wait complete. 2023-07-06T20:13:33.219 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 8176a7ab-1538-4693-a632-a07c7bcd3ad0 4 2023-07-06T20:13:33.245 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:33.244+0000 7fdb245b3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:13:33.252 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:33.250+0000 7fcb0ea3b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:13:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.417+0000 7f998fbff700 1 Processor -- start 2023-07-06T20:13:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.417+0000 7f998fbff700 1 -- start start 2023-07-06T20:13:33.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998fbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b6110 0x7f99880cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998fbff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880cf780 con 0x7f99880b5c90 2023-07-06T20:13:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998fbff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880cd490 con 0x7f998815fca0 2023-07-06T20:13:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998fbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99880cd770 con 0x7f99880b6110 2023-07-06T20:13:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998d99b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f998815fca0 0x7f99880d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:43516/0 (socket says 172.21.15.153:43516) 2023-07-06T20:13:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998d99b700 1 -- 172.21.15.153:0/3334555561 learned_addr learned my addr 172.21.15.153:0/3334555561 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-07-06T20:13:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998e19c700 1 --2- 172.21.15.153:0/3334555561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b6110 0x7f99880cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998c999700 1 -- 172.21.15.153:0/3334555561 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851820471 0 0) 0x7f99880cd490 con 0x7f998815fca0 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998c999700 1 -- 172.21.15.153:0/3334555561 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99740035b0 con 0x7f998815fca0 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998c999700 1 -- 172.21.15.153:0/3334555561 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553309590 0 0) 0x7f99880cf780 con 0x7f99880b5c90 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998e19c700 1 -- 172.21.15.153:0/3334555561 >> v1:172.21.15.153:6789/0 conn(0x7f998815fca0 legacy=0x7f99880d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998e19c700 1 -- 172.21.15.153:0/3334555561 >> v1:172.21.15.27:6789/0 conn(0x7f99880b5c90 legacy=0x7f99880d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.419+0000 7f998e19c700 1 -- 172.21.15.153:0/3334555561 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880cda50 con 0x7f99880b6110 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.420+0000 7f998e19c700 1 --2- 172.21.15.153:0/3334555561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b6110 0x7f99880cfc10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9980009dc0 tx=0x7f998000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=707760f81922ab4b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.420+0000 7f998c999700 1 -- 172.21.15.153:0/3334555561 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000e070 con 0x7f99880b6110 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.420+0000 7f998c999700 1 -- 172.21.15.153:0/3334555561 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000bb20 con 0x7f99880b6110 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.420+0000 7f998c999700 1 -- 172.21.15.153:0/3334555561 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000bed0 con 0x7f99880b6110 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 -- 172.21.15.153:0/3334555561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b6110 msgr2=0x7f99880cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 --2- 172.21.15.153:0/3334555561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b6110 0x7f99880cfc10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9980009dc0 tx=0x7f998000b330 comp rx=0 tx=0).stop 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 -- 172.21.15.153:0/3334555561 shutdown_connections 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 --2- 172.21.15.153:0/3334555561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b6110 0x7f99880cfc10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 -- 172.21.15.153:0/3334555561 >> 172.21.15.153:0/3334555561 conn(0x7f998806c150 msgr2=0x7f998806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:33.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 -- 172.21.15.153:0/3334555561 shutdown_connections 2023-07-06T20:13:33.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.421+0000 7f998fbff700 1 -- 172.21.15.153:0/3334555561 wait complete. 2023-07-06T20:13:33.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.422+0000 7f998fbff700 1 Processor -- start 2023-07-06T20:13:33.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.423+0000 7f998fbff700 1 -- start start 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.423+0000 7f998fbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 0x7f998814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.423+0000 7f998fbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99880b6110 0x7f998815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998fbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998815fca0 0x7f99880aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998fbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998815d780 con 0x7f998815fca0 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998fbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998814f980 con 0x7f99880b6110 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998fbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998815c990 con 0x7f99880b5c90 2023-07-06T20:13:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 0x7f998814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 0x7f998814f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:37908/0 (socket says 172.21.15.153:37908) 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 -- 172.21.15.153:0/4202538795 learned_addr learned my addr 172.21.15.153:0/4202538795 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d19a700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99880b6110 0x7f998815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998e19c700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998815fca0 0x7f99880aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 -- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99880b6110 msgr2=0x7f998815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99880b6110 0x7f998815c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 -- 172.21.15.153:0/4202538795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998815fca0 msgr2=0x7f99880aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998815fca0 0x7f99880aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 -- 172.21.15.153:0/4202538795 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9980009a70 con 0x7f99880b5c90 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.424+0000 7f998d99b700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 0x7f998814f440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f998400c430 tx=0x7f998400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984005d70 con 0x7f99880b5c90 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998400ce40 con 0x7f99880b5c90 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840072f0 con 0x7f99880b5c90 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998815cc70 con 0x7f99880b5c90 2023-07-06T20:13:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998815d990 con 0x7f99880b5c90 2023-07-06T20:13:33.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f998401c440 con 0x7f99880b5c90 2023-07-06T20:13:33.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.425+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f99840077a0 con 0x7f99880b5c90 2023-07-06T20:13:33.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.426+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99880008d0 con 0x7f99880b5c90 2023-07-06T20:13:33.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.430+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9984017070 con 0x7f99880b5c90 2023-07-06T20:13:33.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.514+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8176a7ab-1538-4693-a632-a07c7bcd3ad0", "id": 4} v 0) v1 -- 0x7f998815db80 con 0x7f99880b5c90 2023-07-06T20:13:33.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.521+0000 7f997e7fc700 1 -- 172.21.15.153:0/4202538795 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8176a7ab-1538-4693-a632-a07c7bcd3ad0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f998401c880 con 0x7f99880b5c90 2023-07-06T20:13:33.524 INFO:teuthology.orchestra.run.smithi153.stdout:4 2023-07-06T20:13:33.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 msgr2=0x7f998814f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 0x7f998814f440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f998400c430 tx=0x7f998400c740 comp rx=0 tx=0).stop 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 shutdown_connections 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998815fca0 0x7f99880aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99880b5c90 0x7f998814f440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 --2- 172.21.15.153:0/4202538795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99880b6110 0x7f998815c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 >> 172.21.15.153:0/4202538795 conn(0x7f998806c150 msgr2=0x7f99880b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 shutdown_connections 2023-07-06T20:13:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.523+0000 7f998fbff700 1 -- 172.21.15.153:0/4202538795 wait complete. 2023-07-06T20:13:33.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 3e5a4020-5fa4-4e47-80bd-b03c0c0af152 5 2023-07-06T20:13:33.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.768+0000 7fbab81ff700 1 Processor -- start 2023-07-06T20:13:33.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.769+0000 7fbab81ff700 1 -- start start 2023-07-06T20:13:33.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab81ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbab0159fc0 0x7fbab00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab81ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbab015a390 0x7fbab015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab81ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00cf540 con 0x7fbab00cf0a0 2023-07-06T20:13:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab81ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab00cf820 con 0x7fbab0159fc0 2023-07-06T20:13:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab81ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab00cdcb0 con 0x7fbab015a390 2023-07-06T20:13:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab579a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbab00cf0a0 0x7fbab00b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:43528/0 (socket says 172.21.15.153:43528) 2023-07-06T20:13:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab579a700 1 -- 172.21.15.153:0/451076338 learned_addr learned my addr 172.21.15.153:0/451076338 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-07-06T20:13:33.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab679c700 1 --2- 172.21.15.153:0/451076338 >> v2:172.21.15.153:3300/0 conn(0x7fbab015a390 0x7fbab015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab5f9b700 1 --2- 172.21.15.153:0/451076338 >> v2:172.21.15.27:3300/0 conn(0x7fbab0159fc0 0x7fbab00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab4f99700 1 -- 172.21.15.153:0/451076338 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620437247 0 0) 0x7fbab00cf540 con 0x7fbab00cf0a0 2023-07-06T20:13:33.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab4f99700 1 -- 172.21.15.153:0/451076338 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba9c0035b0 con 0x7fbab00cf0a0 2023-07-06T20:13:33.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.770+0000 7fbab679c700 1 -- 172.21.15.153:0/451076338 >> v2:172.21.15.27:3300/0 conn(0x7fbab0159fc0 msgr2=0x7fbab00d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:33.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab679c700 1 --2- 172.21.15.153:0/451076338 >> v2:172.21.15.27:3300/0 conn(0x7fbab0159fc0 0x7fbab00d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab679c700 1 -- 172.21.15.153:0/451076338 >> v1:172.21.15.153:6789/0 conn(0x7fbab00cf0a0 legacy=0x7fbab00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab679c700 1 -- 172.21.15.153:0/451076338 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00cd660 con 0x7fbab015a390 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab5f9b700 1 --2- 172.21.15.153:0/451076338 >> v2:172.21.15.27:3300/0 conn(0x7fbab0159fc0 0x7fbab00d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab679c700 1 --2- 172.21.15.153:0/451076338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab015d680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbab00b5a60 tx=0x7fbaa800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8338e6af0c808dbb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab4f99700 1 -- 172.21.15.153:0/451076338 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800e070 con 0x7fbab015a390 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab4f99700 1 -- 172.21.15.153:0/451076338 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa800bb20 con 0x7fbab015a390 2023-07-06T20:13:33.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.771+0000 7fbab4f99700 1 -- 172.21.15.153:0/451076338 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8012430 con 0x7fbab015a390 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 -- 172.21.15.153:0/451076338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 msgr2=0x7fbab015d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 --2- 172.21.15.153:0/451076338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab015d680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbab00b5a60 tx=0x7fbaa800b330 comp rx=0 tx=0).stop 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 -- 172.21.15.153:0/451076338 shutdown_connections 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 --2- 172.21.15.153:0/451076338 >> v2:172.21.15.27:3300/0 conn(0x7fbab0159fc0 0x7fbab00d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 --2- 172.21.15.153:0/451076338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab015d680 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 -- 172.21.15.153:0/451076338 >> 172.21.15.153:0/451076338 conn(0x7fbab006c1e0 msgr2=0x7fbab006c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 -- 172.21.15.153:0/451076338 shutdown_connections 2023-07-06T20:13:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.772+0000 7fbab81ff700 1 -- 172.21.15.153:0/451076338 wait complete. 2023-07-06T20:13:33.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.774+0000 7fbab81ff700 1 Processor -- start 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.774+0000 7fbab81ff700 1 -- start start 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab81ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 0x7fbab00cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab81ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbab0159fc0 0x7fbab00b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab81ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab01431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab81ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0152830 con 0x7fbab0159fc0 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab81ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00cb8e0 con 0x7fbab015a390 2023-07-06T20:13:33.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab81ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00caf10 con 0x7fbab00cf0a0 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab5f9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 0x7fbab00cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab679c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab01431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab5f9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 0x7fbab00cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:37920/0 (socket says 172.21.15.153:37920) 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab5f9b700 1 -- 172.21.15.153:0/1143070770 learned_addr learned my addr 172.21.15.153:0/1143070770 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.775+0000 7fbab579a700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbab0159fc0 0x7fbab00b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab679c700 1 -- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 msgr2=0x7fbab00cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab679c700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 0x7fbab00cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab679c700 1 -- 172.21.15.153:0/1143070770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbab0159fc0 msgr2=0x7fbab00b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab679c700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbab0159fc0 0x7fbab00b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab679c700 1 -- 172.21.15.153:0/1143070770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac0094a0 con 0x7fbab015a390 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab5f9b700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 0x7fbab00cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab679c700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab01431b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8016000 tx=0x7fbaa8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8012430 con 0x7fbab015a390 2023-07-06T20:13:33.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa8020070 con 0x7fbab015a390 2023-07-06T20:13:33.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa8009a70 con 0x7fbab015a390 2023-07-06T20:13:33.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.776+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa80098f0 con 0x7fbab015a390 2023-07-06T20:13:33.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.777+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab0152550 con 0x7fbab015a390 2023-07-06T20:13:33.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.777+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbaa8004340 con 0x7fbab015a390 2023-07-06T20:13:33.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.777+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fbaa8022070 con 0x7fbab015a390 2023-07-06T20:13:33.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.778+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab0068190 con 0x7fbab015a390 2023-07-06T20:13:33.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.780+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbaa8003e30 con 0x7fbab015a390 2023-07-06T20:13:33.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.866+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3e5a4020-5fa4-4e47-80bd-b03c0c0af152", "id": 5} v 0) v1 -- 0x7fbab00cb080 con 0x7fbab015a390 2023-07-06T20:13:33.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.871+0000 7fbaa67fc700 1 -- 172.21.15.153:0/1143070770 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e5a4020-5fa4-4e47-80bd-b03c0c0af152", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbaa8004720 con 0x7fbab015a390 2023-07-06T20:13:33.874 INFO:teuthology.orchestra.run.smithi153.stdout:5 2023-07-06T20:13:33.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 msgr2=0x7fbab01431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:33.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab01431b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8016000 tx=0x7fbaa8004ad0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 shutdown_connections 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab015a390 0x7fbab01431b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbab00cf0a0 0x7fbab00cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 --2- 172.21.15.153:0/1143070770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbab0159fc0 0x7fbab00b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 >> 172.21.15.153:0/1143070770 conn(0x7fbab006c1e0 msgr2=0x7fbab00cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 shutdown_connections 2023-07-06T20:13:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:33.873+0000 7fbab81ff700 1 -- 172.21.15.153:0/1143070770 wait complete. 2023-07-06T20:13:33.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new ad6e9be7-c296-45db-a17f-ebb2600dc3f3 6 2023-07-06T20:13:34.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.091+0000 7f9be81ff700 1 Processor -- start 2023-07-06T20:13:34.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.091+0000 7f9be81ff700 1 -- start start 2023-07-06T20:13:34.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be81ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9be0159fc0 0x7f9be00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be81ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9be00cf0a0 0x7f9be015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be81ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be015a390 0x7f9be00b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be81ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be00cf820 con 0x7f9be00cf0a0 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be81ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be00cdcb0 con 0x7f9be0159fc0 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be81ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be015dbc0 con 0x7f9be015a390 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be5f9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9be0159fc0 0x7f9be00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be679c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be015a390 0x7f9be00b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be5f9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9be0159fc0 0x7f9be00d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33186/0 (socket says 172.21.15.153:33186) 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be5f9b700 1 -- 172.21.15.153:0/4236082918 learned_addr learned my addr 172.21.15.153:0/4236082918 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:34.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.092+0000 7f9be579a700 1 --2- 172.21.15.153:0/4236082918 >> v2:172.21.15.27:3300/0 conn(0x7f9be00cf0a0 0x7f9be015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.097 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.093+0000 7fdb245b3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:13:34.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be5f9b700 1 -- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be015a390 msgr2=0x7f9be00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:34.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be5f9b700 1 --2- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be015a390 0x7f9be00b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be5f9b700 1 -- 172.21.15.153:0/4236082918 >> v2:172.21.15.27:3300/0 conn(0x7f9be00cf0a0 msgr2=0x7f9be015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be5f9b700 1 --2- 172.21.15.153:0/4236082918 >> v2:172.21.15.27:3300/0 conn(0x7f9be00cf0a0 0x7f9be015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be5f9b700 1 -- 172.21.15.153:0/4236082918 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00cf540 con 0x7f9be0159fc0 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be679c700 1 --2- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be015a390 0x7f9be00b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be5f9b700 1 --2- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00d29b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9be006ce50 tx=0x7f9bdc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e027dcd331a0dd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be4f99700 1 -- 172.21.15.153:0/4236082918 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc011830 con 0x7f9be0159fc0 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be4f99700 1 -- 172.21.15.153:0/4236082918 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc011990 con 0x7f9be0159fc0 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.093+0000 7f9be4f99700 1 -- 172.21.15.153:0/4236082918 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc011d20 con 0x7f9be0159fc0 2023-07-06T20:13:34.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 -- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 msgr2=0x7f9be00d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 --2- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00d29b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9be006ce50 tx=0x7f9bdc009bd0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 -- 172.21.15.153:0/4236082918 shutdown_connections 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 --2- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be015a390 0x7f9be00b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 --2- 172.21.15.153:0/4236082918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00d29b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 --2- 172.21.15.153:0/4236082918 >> v2:172.21.15.27:3300/0 conn(0x7f9be00cf0a0 0x7f9be015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 -- 172.21.15.153:0/4236082918 >> 172.21.15.153:0/4236082918 conn(0x7f9be006c1e0 msgr2=0x7f9be006c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 -- 172.21.15.153:0/4236082918 shutdown_connections 2023-07-06T20:13:34.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.094+0000 7f9be81ff700 1 -- 172.21.15.153:0/4236082918 wait complete. 2023-07-06T20:13:34.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.096+0000 7f9be81ff700 1 Processor -- start 2023-07-06T20:13:34.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.096+0000 7f9be81ff700 1 -- start start 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be81ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be00cf0a0 0x7f9be015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be81ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be81ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be015a390 0x7f9be00b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be81ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00b78e0 con 0x7f9be015a390 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be81ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00b72f0 con 0x7f9be0159fc0 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be81ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00cbd60 con 0x7f9be00cf0a0 2023-07-06T20:13:34.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be5f9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be00cf0a0 0x7f9be015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be579a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be5f9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be00cf0a0 0x7f9be015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:37948/0 (socket says 172.21.15.153:37948) 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be5f9b700 1 -- 172.21.15.153:0/1789821436 learned_addr learned my addr 172.21.15.153:0/1789821436 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.097+0000 7f9be679c700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be015a390 0x7f9be00b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be579a700 1 -- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be00cf0a0 msgr2=0x7f9be015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be579a700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be00cf0a0 0x7f9be015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be579a700 1 -- 172.21.15.153:0/1789821436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be015a390 msgr2=0x7f9be00b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be579a700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be015a390 0x7f9be00b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be579a700 1 -- 172.21.15.153:0/1789821436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8009a70 con 0x7f9be0159fc0 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be679c700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be015a390 0x7f9be00b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be579a700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00c9040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e870 tx=0x7f9bd000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0013680 con 0x7f9be0159fc0 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd00137e0 con 0x7f9be0159fc0 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc009550 con 0x7f9be0159fc0 2023-07-06T20:13:34.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0013b50 con 0x7f9be0159fc0 2023-07-06T20:13:34.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.098+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be00b7570 con 0x7f9be0159fc0 2023-07-06T20:13:34.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.099+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9bd0005a20 con 0x7f9be0159fc0 2023-07-06T20:13:34.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.099+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f9bd0005cd0 con 0x7f9be0159fc0 2023-07-06T20:13:34.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.100+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be00cc280 con 0x7f9be0159fc0 2023-07-06T20:13:34.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.102+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9bd0021060 con 0x7f9be0159fc0 2023-07-06T20:13:34.124 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.121+0000 7fcb0ea3b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:13:34.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.188+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad6e9be7-c296-45db-a17f-ebb2600dc3f3", "id": 6} v 0) v1 -- 0x7f9be0068190 con 0x7f9be0159fc0 2023-07-06T20:13:34.191 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.189+0000 7fdb245b3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:13:34.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.192+0000 7f9bd67fc700 1 -- 172.21.15.153:0/1789821436 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad6e9be7-c296-45db-a17f-ebb2600dc3f3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9bd0029020 con 0x7f9be0159fc0 2023-07-06T20:13:34.195 INFO:teuthology.orchestra.run.smithi153.stdout:6 2023-07-06T20:13:34.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 msgr2=0x7f9be00c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00c9040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e870 tx=0x7f9bd000eb80 comp rx=0 tx=0).stop 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 shutdown_connections 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9be015a390 0x7f9be00b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9be00cf0a0 0x7f9be015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 --2- 172.21.15.153:0/1789821436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9be0159fc0 0x7f9be00c9040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.193+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 >> 172.21.15.153:0/1789821436 conn(0x7f9be006c1e0 msgr2=0x7f9be00b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.194+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 shutdown_connections 2023-07-06T20:13:34.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.194+0000 7f9be81ff700 1 -- 172.21.15.153:0/1789821436 wait complete. 2023-07-06T20:13:34.213 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f 7 2023-07-06T20:13:34.221 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.218+0000 7fcb0ea3b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:13:34.368 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.367+0000 7fdb245b3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:13:34.396 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.394+0000 7fcb0ea3b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:13:34.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.402+0000 7f8d75bff700 1 Processor -- start 2023-07-06T20:13:34.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.402+0000 7f8d75bff700 1 -- start start 2023-07-06T20:13:34.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d75bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d7015fca0 0x7f8d700d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d75bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d700b6110 0x7f8d700cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d75bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700cf780 con 0x7f8d700b5c90 2023-07-06T20:13:34.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d75bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700cd490 con 0x7f8d7015fca0 2023-07-06T20:13:34.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d75bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700cd770 con 0x7f8d700b6110 2023-07-06T20:13:34.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d6ffff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d700b6110 0x7f8d700cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.403+0000 7f8d6f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d7015fca0 0x7f8d700d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d7015fca0 0x7f8d700d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.153:49578/0 (socket says 172.21.15.153:49578) 2023-07-06T20:13:34.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/2828911960 learned_addr learned my addr 172.21.15.153:0/2828911960 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:34.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6effd700 1 --1- 172.21.15.153:0/2828911960 >> v1:172.21.15.27:6789/0 conn(0x7f8d700b5c90 0x7f8d700d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.153:55746/0 (socket says 172.21.15.153:55746) 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/2828911960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d700b6110 msgr2=0x7f8d700cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 --2- 172.21.15.153:0/2828911960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d700b6110 0x7f8d700cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/2828911960 >> v1:172.21.15.27:6789/0 conn(0x7f8d700b5c90 legacy=0x7f8d700d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/2828911960 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700cda50 con 0x7f8d7015fca0 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6ffff700 1 --2- 172.21.15.153:0/2828911960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d700b6110 0x7f8d700cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6e7fc700 1 -- 172.21.15.153:0/2828911960 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759312497 0 0) 0x7f8d700cf780 con 0x7f8d700b5c90 2023-07-06T20:13:34.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6f7fe700 1 --2- 172.21.15.153:0/2828911960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d7015fca0 0x7f8d700d1420 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098f0 tx=0x7f8d60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b922246c7c84d7f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.404+0000 7f8d6e7fc700 1 -- 172.21.15.153:0/2828911960 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011830 con 0x7f8d7015fca0 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d6e7fc700 1 -- 172.21.15.153:0/2828911960 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60011990 con 0x7f8d7015fca0 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d6e7fc700 1 -- 172.21.15.153:0/2828911960 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011d20 con 0x7f8d7015fca0 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 -- 172.21.15.153:0/2828911960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d7015fca0 msgr2=0x7f8d700d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 --2- 172.21.15.153:0/2828911960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d7015fca0 0x7f8d700d1420 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098f0 tx=0x7f8d60009c00 comp rx=0 tx=0).stop 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 -- 172.21.15.153:0/2828911960 shutdown_connections 2023-07-06T20:13:34.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 --2- 172.21.15.153:0/2828911960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d700b6110 0x7f8d700cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 --2- 172.21.15.153:0/2828911960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d7015fca0 0x7f8d700d1420 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 -- 172.21.15.153:0/2828911960 >> 172.21.15.153:0/2828911960 conn(0x7f8d7006c150 msgr2=0x7f8d7006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:34.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 -- 172.21.15.153:0/2828911960 shutdown_connections 2023-07-06T20:13:34.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.405+0000 7f8d75bff700 1 -- 172.21.15.153:0/2828911960 wait complete. 2023-07-06T20:13:34.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.407+0000 7f8d75bff700 1 Processor -- start 2023-07-06T20:13:34.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.407+0000 7f8d75bff700 1 -- start start 2023-07-06T20:13:34.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d75bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 0x7f8d700c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d75bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d700b6110 0x7f8d70149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d75bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d7015fca0 0x7f8d700c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d75bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c74a0 con 0x7f8d700b6110 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d75bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c7780 con 0x7f8d700b5c90 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d75bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c7a60 con 0x7f8d7015fca0 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 0x7f8d700c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 0x7f8d700c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33208/0 (socket says 172.21.15.153:33208) 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/3715934809 learned_addr learned my addr 172.21.15.153:0/3715934809 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d6ffff700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d7015fca0 0x7f8d700c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.408+0000 7f8d6effd700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d700b6110 0x7f8d70149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d7015fca0 msgr2=0x7f8d700c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6f7fe700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d7015fca0 0x7f8d700c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/3715934809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d700b6110 msgr2=0x7f8d70149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6f7fe700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d700b6110 0x7f8d70149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6f7fe700 1 -- 172.21.15.153:0/3715934809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68009a70 con 0x7f8d700b5c90 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6ffff700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d7015fca0 0x7f8d700c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d6f7fe700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 0x7f8d700c8540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009c40 tx=0x7f8d60004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:34.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600044d0 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6001e070 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600038c0 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.409+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d60009550 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.410+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700c8020 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.410+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8d60003d20 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.410+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8d6001adc0 con 0x7f8d700b5c90 2023-07-06T20:13:34.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.410+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700c8210 con 0x7f8d700b5c90 2023-07-06T20:13:34.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.413+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8d60020070 con 0x7f8d700b5c90 2023-07-06T20:13:34.463 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.461+0000 7fdb245b3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:13:34.488 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.486+0000 7fcb0ea3b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:13:34.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.499+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f", "id": 7} v 0) v1 -- 0x7f8d700008d0 con 0x7f8d700b5c90 2023-07-06T20:13:34.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.505+0000 7f8d57fff700 1 -- 172.21.15.153:0/3715934809 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8d60015020 con 0x7f8d700b5c90 2023-07-06T20:13:34.508 INFO:teuthology.orchestra.run.smithi153.stdout:7 2023-07-06T20:13:34.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 msgr2=0x7f8d700c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 0x7f8d700c8540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009c40 tx=0x7f8d60004750 comp rx=0 tx=0).stop 2023-07-06T20:13:34.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 shutdown_connections 2023-07-06T20:13:34.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8d7015fca0 0x7f8d700c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d700b5c90 0x7f8d700c8540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 --2- 172.21.15.153:0/3715934809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d700b6110 0x7f8d70149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 >> 172.21.15.153:0/3715934809 conn(0x7f8d7006c150 msgr2=0x7f8d700b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:34.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 shutdown_connections 2023-07-06T20:13:34.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-07-06T20:13:34.506+0000 7f8d75bff700 1 -- 172.21.15.153:0/3715934809 wait complete. 2023-07-06T20:13:34.526 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-06T20:13:34.526 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-06T20:13:34.530 INFO:tasks.ceph.osd.0:Started 2023-07-06T20:13:34.531 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-06T20:13:34.531 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-06T20:13:34.535 INFO:tasks.ceph.osd.1:Started 2023-07-06T20:13:34.535 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-06T20:13:34.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-06T20:13:34.539 INFO:tasks.ceph.osd.2:Started 2023-07-06T20:13:34.539 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-06T20:13:34.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-06T20:13:34.545 INFO:tasks.ceph.osd.3:Started 2023-07-06T20:13:34.545 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-06T20:13:34.546 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-06T20:13:34.569 INFO:tasks.ceph.osd.4:Started 2023-07-06T20:13:34.570 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-06T20:13:34.570 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-06T20:13:34.574 INFO:tasks.ceph.osd.5:Started 2023-07-06T20:13:34.574 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-06T20:13:34.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-06T20:13:34.579 INFO:tasks.ceph.osd.6:Started 2023-07-06T20:13:34.580 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-06T20:13:34.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-06T20:13:34.584 INFO:tasks.ceph.osd.7:Started 2023-07-06T20:13:34.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:13:34.757 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.755+0000 7fdb245b3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:13:34.760 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.758+0000 7fcb0ea3b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:13:34.863 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.862+0000 7fdb245b3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:13:34.865 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.862+0000 7fcb0ea3b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:13:34.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.865+0000 7f78cf7ff700 1 Processor -- start 2023-07-06T20:13:34.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.865+0000 7f78cf7ff700 1 -- start start 2023-07-06T20:13:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.866+0000 7f78cf7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.866+0000 7f78cf7ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78c80b60e0 0x7f78c80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.866+0000 7f78cf7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c80cf780 con 0x7f78c815fca0 2023-07-06T20:13:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.866+0000 7f78cf7ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c80cd490 con 0x7f78c80b60e0 2023-07-06T20:13:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.866+0000 7f78cf7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80cd770 con 0x7f78c80b5c90 2023-07-06T20:13:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78cdd9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78c80b60e0 0x7f78c80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78ccd9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78cd59b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f78c815fca0 0x7f78c80d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:35064/0 (socket says 172.21.15.27:35064) 2023-07-06T20:13:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78cd59b700 1 -- 172.21.15.27:0/929540333 learned_addr learned my addr 172.21.15.27:0/929540333 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78bffff700 1 -- 172.21.15.27:0/929540333 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035879921 0 0) 0x7f78c80cf780 con 0x7f78c815fca0 2023-07-06T20:13:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78bffff700 1 -- 172.21.15.27:0/929540333 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b40035b0 con 0x7f78c815fca0 2023-07-06T20:13:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78ccd9a700 1 -- 172.21.15.27:0/929540333 >> v2:172.21.15.153:3300/0 conn(0x7f78c80b60e0 msgr2=0x7f78c80d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78ccd9a700 1 --2- 172.21.15.27:0/929540333 >> v2:172.21.15.153:3300/0 conn(0x7f78c80b60e0 0x7f78c80d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78ccd9a700 1 -- 172.21.15.27:0/929540333 >> v1:172.21.15.153:6789/0 conn(0x7f78c815fca0 legacy=0x7f78c80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78ccd9a700 1 -- 172.21.15.27:0/929540333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80cda50 con 0x7f78c80b5c90 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.867+0000 7f78cdd9c700 1 --2- 172.21.15.27:0/929540333 >> v2:172.21.15.153:3300/0 conn(0x7f78c80b60e0 0x7f78c80d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.868+0000 7f78ccd9a700 1 --2- 172.21.15.27:0/929540333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80d2d40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009840 tx=0x7f78b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634c6307cac324c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.868+0000 7f78bffff700 1 -- 172.21.15.27:0/929540333 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8011830 con 0x7f78c80b5c90 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.868+0000 7f78bffff700 1 -- 172.21.15.27:0/929540333 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8011990 con 0x7f78c80b5c90 2023-07-06T20:13:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.868+0000 7f78bffff700 1 -- 172.21.15.27:0/929540333 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8011d20 con 0x7f78c80b5c90 2023-07-06T20:13:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 -- 172.21.15.27:0/929540333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 msgr2=0x7f78c80d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/929540333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80d2d40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009840 tx=0x7f78b8009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 -- 172.21.15.27:0/929540333 shutdown_connections 2023-07-06T20:13:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/929540333 >> v2:172.21.15.153:3300/0 conn(0x7f78c80b60e0 0x7f78c80d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/929540333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80d2d40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 -- 172.21.15.27:0/929540333 >> 172.21.15.27:0/929540333 conn(0x7f78c806c150 msgr2=0x7f78c806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 -- 172.21.15.27:0/929540333 shutdown_connections 2023-07-06T20:13:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.869+0000 7f78cf7ff700 1 -- 172.21.15.27:0/929540333 wait complete. 2023-07-06T20:13:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.871+0000 7f78cf7ff700 1 Processor -- start 2023-07-06T20:13:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.872+0000 7f78cf7ff700 1 -- start start 2023-07-06T20:13:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cf7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cf7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 0x7f78c8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cf7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c815fca0 0x7f78c80a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cf7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80b4b00 con 0x7f78c80b60e0 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cf7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80b3ef0 con 0x7f78c815fca0 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cf7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80b41d0 con 0x7f78c80b5c90 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78ccd9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 0x7f78c8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cd59b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78ccd9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 0x7f78c8159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42662/0 (socket says 172.21.15.27:42662) 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78ccd9a700 1 -- 172.21.15.27:0/3168963547 learned_addr learned my addr 172.21.15.27:0/3168963547 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78cdd9c700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c815fca0 0x7f78c80a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78ccd9a700 1 -- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 msgr2=0x7f78c80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.873+0000 7f78ccd9a700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78ccd9a700 1 -- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c815fca0 msgr2=0x7f78c80a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78ccd9a700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c815fca0 0x7f78c80a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78ccd9a700 1 -- 172.21.15.27:0/3168963547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c0009a70 con 0x7f78c80b60e0 2023-07-06T20:13:34.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78ccd9a700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 0x7f78c8159340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78b80090b0 tx=0x7f78b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:34.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8004550 con 0x7f78c80b60e0 2023-07-06T20:13:34.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b801f070 con 0x7f78c80b60e0 2023-07-06T20:13:34.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b80094a0 con 0x7f78c80b60e0 2023-07-06T20:13:34.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b80038e0 con 0x7f78c80b60e0 2023-07-06T20:13:34.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.874+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c80b4790 con 0x7f78c80b60e0 2023-07-06T20:13:34.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.875+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f78b8003d40 con 0x7f78c80b60e0 2023-07-06T20:13:34.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.875+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f78b8021070 con 0x7f78c80b60e0 2023-07-06T20:13:34.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.875+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c80b4920 con 0x7f78c80b60e0 2023-07-06T20:13:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.878+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f78b8045b00 con 0x7f78c80b60e0 2023-07-06T20:13:34.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.951+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78c80b4c70 con 0x7f78c80b60e0 2023-07-06T20:13:34.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.952+0000 7f78bdffb700 1 -- 172.21.15.27:0/3168963547 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f78b8034030 con 0x7f78c80b60e0 2023-07-06T20:13:34.954 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:13:34.954 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:34.501804+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:13:34.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 msgr2=0x7f78c8159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:34.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 0x7f78c8159340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78b80090b0 tx=0x7f78b8004750 comp rx=0 tx=0).stop 2023-07-06T20:13:34.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 shutdown_connections 2023-07-06T20:13:34.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c815fca0 0x7f78c80a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78c80b5c90 0x7f78c80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 --2- 172.21.15.27:0/3168963547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78c80b60e0 0x7f78c8159340 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:34.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 >> 172.21.15.27:0/3168963547 conn(0x7f78c806c150 msgr2=0x7f78c80cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:34.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 shutdown_connections 2023-07-06T20:13:34.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:34.953+0000 7f78cf7ff700 1 -- 172.21.15.27:0/3168963547 wait complete. 2023-07-06T20:13:34.964 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:34.961+0000 7fcb0ea3b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:13:34.969 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:34.968+0000 7fdb245b3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:13:34.972 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-06T20:13:34.973 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-06T20:13:35.008 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-07-06T20:13:35.007+0000 7f35d139e640 -1 Falling back to public interface 2023-07-06T20:13:35.074 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:35.071+0000 7fcb0ea3b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:13:35.080 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:35.078+0000 7fdb245b3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:13:35.267 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-07-06T20:13:35.265+0000 7f25bedaf640 -1 Falling back to public interface 2023-07-06T20:13:35.267 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-07-06T20:13:35.265+0000 7f1c2aa98640 -1 Falling back to public interface 2023-07-06T20:13:35.268 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-07-06T20:13:35.265+0000 7fac4af02640 -1 Falling back to public interface 2023-07-06T20:13:35.268 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-07-06T20:13:35.266+0000 7fd078763640 -1 Falling back to public interface 2023-07-06T20:13:35.269 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-07-06T20:13:35.266+0000 7fac3cdfd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6806/0: (98) Address already in use 2023-07-06T20:13:35.286 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-07-06T20:13:35.284+0000 7ff645c47640 -1 Falling back to public interface 2023-07-06T20:13:35.290 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-07-06T20:13:35.289+0000 7fa414d1a640 -1 Falling back to public interface 2023-07-06T20:13:35.303 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:35.300+0000 7fcb0ea3b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:13:35.308 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-07-06T20:13:35.306+0000 7f5500868640 -1 Falling back to public interface 2023-07-06T20:13:35.320 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:35.318+0000 7fdb245b3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:13:35.413 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:35.410+0000 7fcb0ea3b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:13:35.425 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:35.424+0000 7fdb245b3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:13:35.770 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:35.768+0000 7fcb0ea3b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:13:35.779 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T20:13:35.791 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:35.790+0000 7fdb245b3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:13:35.862 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:35.859+0000 7fcb0ea3b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:13:35.890 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:35.889+0000 7fdb245b3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:13:35.962 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.960+0000 7fc3be7ff700 1 Processor -- start 2023-07-06T20:13:35.962 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.960+0000 7fc3be7ff700 1 -- start start 2023-07-06T20:13:35.964 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3be7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc3b80b5c90 0x7fc3b80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:35.964 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3be7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b80b6110 0x7fc3b80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:35.964 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3be7ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b80cf780 con 0x7fc3b815fca0 2023-07-06T20:13:35.964 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3be7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b80cd490 con 0x7fc3b80b5c90 2023-07-06T20:13:35.964 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3be7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80cd770 con 0x7fc3b80b6110 2023-07-06T20:13:35.964 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3b77fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc3b80b5c90 0x7fc3b80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:35.965 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3b7fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc3b815fca0 0x7fc3b80d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60548/0 (socket says 172.21.15.27:60548) 2023-07-06T20:13:35.965 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3b7fff700 1 -- 172.21.15.27:0/539856519 learned_addr learned my addr 172.21.15.27:0/539856519 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:35.965 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.962+0000 7fc3bcd9c700 1 --2- 172.21.15.27:0/539856519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b80b6110 0x7fc3b80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:35.966 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b77fe700 1 -- 172.21.15.27:0/539856519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b80b6110 msgr2=0x7fc3b80d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:35.966 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b77fe700 1 --2- 172.21.15.27:0/539856519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b80b6110 0x7fc3b80d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:35.966 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b77fe700 1 -- 172.21.15.27:0/539856519 >> v1:172.21.15.27:6789/0 conn(0x7fc3b815fca0 legacy=0x7fc3b80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b77fe700 1 -- 172.21.15.27:0/539856519 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b80cda50 con 0x7fc3b80b5c90 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b6ffd700 1 -- 172.21.15.27:0/539856519 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359348935 0 0) 0x7fc3b80cf780 con 0x7fc3b815fca0 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b77fe700 1 --2- 172.21.15.27:0/539856519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b80d2d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009840 tx=0x7fc3ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66fb2e1e4d72a3f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b6ffd700 1 -- 172.21.15.27:0/539856519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac011830 con 0x7fc3b80b5c90 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b6ffd700 1 -- 172.21.15.27:0/539856519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac011990 con 0x7fc3b80b5c90 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.963+0000 7fc3b6ffd700 1 -- 172.21.15.27:0/539856519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac011d20 con 0x7fc3b80b5c90 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 -- 172.21.15.27:0/539856519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 msgr2=0x7fc3b80d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:35.967 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/539856519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b80d2d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009840 tx=0x7fc3ac009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:35.968 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 -- 172.21.15.27:0/539856519 shutdown_connections 2023-07-06T20:13:35.968 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/539856519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b80b6110 0x7fc3b80d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:35.968 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/539856519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b80d2d40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:35.968 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 -- 172.21.15.27:0/539856519 >> 172.21.15.27:0/539856519 conn(0x7fc3b806c150 msgr2=0x7fc3b806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:35.968 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 -- 172.21.15.27:0/539856519 shutdown_connections 2023-07-06T20:13:35.968 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.964+0000 7fc3be7ff700 1 -- 172.21.15.27:0/539856519 wait complete. 2023-07-06T20:13:35.970 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.966+0000 7fc3be7ff700 1 Processor -- start 2023-07-06T20:13:35.970 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.966+0000 7fc3be7ff700 1 -- start start 2023-07-06T20:13:35.970 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3be7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b81f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:35.971 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3be7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b80b6110 0x7fc3b80b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:35.971 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3be7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b815fca0 0x7fc3b80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:35.971 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3be7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b81551f0 con 0x7fc3b80b5c90 2023-07-06T20:13:35.971 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3be7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b81f60f0 con 0x7fc3b80b6110 2023-07-06T20:13:35.971 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3be7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8155440 con 0x7fc3b815fca0 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3b7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b81f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3b7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b81f1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42684/0 (socket says 172.21.15.27:42684) 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3b7fff700 1 -- 172.21.15.27:0/3674920298 learned_addr learned my addr 172.21.15.27:0/3674920298 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.967+0000 7fc3b77fe700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b80b6110 0x7fc3b80b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3bcd9c700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b815fca0 0x7fc3b80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b7fff700 1 -- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b815fca0 msgr2=0x7fc3b80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:35.972 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b7fff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b815fca0 0x7fc3b80b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b7fff700 1 -- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b80b6110 msgr2=0x7fc3b80b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b7fff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b80b6110 0x7fc3b80b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b7fff700 1 -- 172.21.15.27:0/3674920298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac0094a0 con 0x7fc3b80b5c90 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b7fff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b81f1550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c550 tx=0x7fc3a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8004c80 con 0x7fc3b80b5c90 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8004e00 con 0x7fc3b80b5c90 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8007310 con 0x7fc3b80b5c90 2023-07-06T20:13:35.973 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.968+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b8155720 con 0x7fc3b80b5c90 2023-07-06T20:13:35.974 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.969+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b81f1a90 con 0x7fc3b80b5c90 2023-07-06T20:13:35.974 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.969+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc3a80133c0 con 0x7fc3b80b5c90 2023-07-06T20:13:35.974 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.969+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc3a8013d40 con 0x7fc3b80b5c90 2023-07-06T20:13:35.974 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.970+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b8155c40 con 0x7fc3b80b5c90 2023-07-06T20:13:35.975 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:35.972+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc3b8155c40 con 0x7fc3b80b5c90 2023-07-06T20:13:36.049 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.047+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3b80008d0 con 0x7fc3b80b5c90 2023-07-06T20:13:36.049 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.048+0000 7fc3b4ff9700 1 -- 172.21.15.27:0/3674920298 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc3a8013720 con 0x7fc3b80b5c90 2023-07-06T20:13:36.050 INFO:teuthology.misc.health.smithi027.stdout: 2023-07-06T20:13:36.050 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":10,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:34.501804+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 msgr2=0x7fc3b81f1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b81f1550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c550 tx=0x7fc3a800c860 comp rx=0 tx=0).stop 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 shutdown_connections 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc3b815fca0 0x7fc3b80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b80b5c90 0x7fc3b81f1550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 --2- 172.21.15.27:0/3674920298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b80b6110 0x7fc3b80b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 >> 172.21.15.27:0/3674920298 conn(0x7fc3b806c150 msgr2=0x7fc3b80b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:36.052 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 shutdown_connections 2023-07-06T20:13:36.052 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:36.049+0000 7fc3be7ff700 1 -- 172.21.15.27:0/3674920298 wait complete. 2023-07-06T20:13:36.065 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T20:13:36.490 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:36.488+0000 7fcb0ea3b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:13:36.550 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:36.548+0000 7fdb245b3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:13:36.875 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:36.873+0000 7fcb0ea3b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:13:36.949 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:36.947+0000 7fdb245b3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:13:37.058 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:37.055+0000 7fcb0ea3b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:13:37.136 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:37.134+0000 7fdb245b3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:13:37.502 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:37.499+0000 7fcb0ea3b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:13:37.614 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:37.612+0000 7fdb245b3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:13:37.684 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:37.681+0000 7fcb0ea3b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:13:37.842 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:37.841+0000 7fdb245b3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:13:37.887 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:37.884+0000 7fcb0ea3b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:13:38.076 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:38.074+0000 7fdb245b3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:13:38.324 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:38.323+0000 7fcb0ea3b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:13:38.501 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-07-06T20:13:38.500+0000 7fdb245b3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:13:38.578 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-07-06T20:13:38.576+0000 7f35d139e640 -1 osd.0 0 log_to_monitors true 2023-07-06T20:13:38.782 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-07-06T20:13:38.780+0000 7fd078763640 -1 osd.5 0 log_to_monitors true 2023-07-06T20:13:38.794 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-07-06T20:13:38.792+0000 7f1c2aa98640 -1 osd.3 0 log_to_monitors true 2023-07-06T20:13:38.829 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-07-06T20:13:38.827+0000 7fac4af02640 -1 osd.4 0 log_to_monitors true 2023-07-06T20:13:38.878 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-07-06T20:13:38.877+0000 7fa414d1a640 -1 osd.1 0 log_to_monitors true 2023-07-06T20:13:38.888 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-07-06T20:13:38.886+0000 7f25bedaf640 -1 osd.2 0 log_to_monitors true 2023-07-06T20:13:39.098 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-07-06T20:13:39.097+0000 7ff645c47640 -1 osd.7 0 log_to_monitors true 2023-07-06T20:13:39.128 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-07-06T20:13:39.126+0000 7f5500868640 -1 osd.6 0 log_to_monitors true 2023-07-06T20:13:39.346 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:39.344+0000 7fcac83fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-06T20:13:39.346 INFO:tasks.ceph.mgr.x.smithi153.stderr:2023-07-06T20:13:39.345+0000 7fcac83fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-06T20:13:42.472 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-07-06T20:13:42.469+0000 7fac3adf9700 -1 osd.4 0 waiting for initial osdmap 2023-07-06T20:13:42.499 INFO:tasks.ceph.osd.4.smithi153.stderr:2023-07-06T20:13:42.497+0000 7fac315fb700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.524 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-07-06T20:13:42.522+0000 7f25ad3f6700 -1 osd.2 0 waiting for initial osdmap 2023-07-06T20:13:42.534 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-07-06T20:13:42.533+0000 7fa4033f6700 -1 osd.1 0 waiting for initial osdmap 2023-07-06T20:13:42.538 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-07-06T20:13:42.536+0000 7f25a53fb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.550 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-07-06T20:13:42.549+0000 7fa3fb3fb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.555 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-07-06T20:13:42.553+0000 7f35bf9f6700 -1 osd.0 0 waiting for initial osdmap 2023-07-06T20:13:42.566 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-07-06T20:13:42.565+0000 7f35b79fb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.609 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-07-06T20:13:42.608+0000 7f1c191f6700 -1 osd.3 0 waiting for initial osdmap 2023-07-06T20:13:42.627 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-07-06T20:13:42.626+0000 7f1c111fb700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.821 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-07-06T20:13:42.819+0000 7f54eeff6700 -1 osd.6 0 waiting for initial osdmap 2023-07-06T20:13:42.837 INFO:tasks.ceph.osd.6.smithi153.stderr:2023-07-06T20:13:42.835+0000 7f54e6ffb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.852 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-07-06T20:13:42.850+0000 7fd066df6700 -1 osd.5 0 waiting for initial osdmap 2023-07-06T20:13:42.863 INFO:tasks.ceph.osd.5.smithi153.stderr:2023-07-06T20:13:42.862+0000 7fd05edfb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:42.870 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T20:13:43.049 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.048+0000 7f041e1ff700 1 Processor -- start 2023-07-06T20:13:43.050 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.048+0000 7f041e1ff700 1 -- start start 2023-07-06T20:13:43.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f041e1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04180b5c90 0x7f04180d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:43.051 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f041e1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04180cf780 con 0x7f041815fca0 2023-07-06T20:13:43.052 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f041e1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04180cd490 con 0x7f04180b6110 2023-07-06T20:13:43.052 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f041e1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04180cd770 con 0x7f04180b5c90 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f04177fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f041815fca0 0x7f04180d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43432/0 (socket says 172.21.15.27:43432) 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f0416ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04180b5c90 0x7f04180d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f04177fe700 1 -- 172.21.15.27:0/418001453 learned_addr learned my addr 172.21.15.27:0/418001453 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f04167fc700 1 -- 172.21.15.27:0/418001453 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964159899 0 0) 0x7f04180cf780 con 0x7f041815fca0 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f0416ffd700 1 -- 172.21.15.27:0/418001453 >> v1:172.21.15.153:6789/0 conn(0x7f04180b6110 legacy=0x7f04180cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f0416ffd700 1 -- 172.21.15.27:0/418001453 >> v1:172.21.15.27:6789/0 conn(0x7f041815fca0 legacy=0x7f04180d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.053 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.050+0000 7f0416ffd700 1 -- 172.21.15.27:0/418001453 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180cda50 con 0x7f04180b5c90 2023-07-06T20:13:43.054 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.051+0000 7f0416ffd700 1 --2- 172.21.15.27:0/418001453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04180b5c90 0x7f04180d2d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f040c009840 tx=0x7f040c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db34e09b6c4c18b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:43.054 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.051+0000 7f04167fc700 1 -- 172.21.15.27:0/418001453 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c011830 con 0x7f04180b5c90 2023-07-06T20:13:43.054 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.051+0000 7f04167fc700 1 -- 172.21.15.27:0/418001453 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c011990 con 0x7f04180b5c90 2023-07-06T20:13:43.054 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.051+0000 7f04167fc700 1 -- 172.21.15.27:0/418001453 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c011d20 con 0x7f04180b5c90 2023-07-06T20:13:43.055 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.051+0000 7f041e1ff700 1 -- 172.21.15.27:0/418001453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04180b5c90 msgr2=0x7f04180d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.055 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.051+0000 7f041e1ff700 1 --2- 172.21.15.27:0/418001453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04180b5c90 0x7f04180d2d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f040c009840 tx=0x7f040c009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:43.055 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.052+0000 7f041e1ff700 1 -- 172.21.15.27:0/418001453 shutdown_connections 2023-07-06T20:13:43.055 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.052+0000 7f041e1ff700 1 --2- 172.21.15.27:0/418001453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04180b5c90 0x7f04180d2d40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.055 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.052+0000 7f041e1ff700 1 -- 172.21.15.27:0/418001453 >> 172.21.15.27:0/418001453 conn(0x7f041806c150 msgr2=0x7f041806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:43.055 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.052+0000 7f041e1ff700 1 -- 172.21.15.27:0/418001453 shutdown_connections 2023-07-06T20:13:43.056 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.052+0000 7f041e1ff700 1 -- 172.21.15.27:0/418001453 wait complete. 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.054+0000 7f041e1ff700 1 Processor -- start 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.054+0000 7f041e1ff700 1 -- start start 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f041e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 0x7f041815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f041e1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04180b6110 0x7f04181f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f041e1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f041815fca0 0x7f04181f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f041e1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04181f1b50 con 0x7f041815fca0 2023-07-06T20:13:43.057 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f041e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04181f60d0 con 0x7f04180b5c90 2023-07-06T20:13:43.058 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f041e1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04181f5b10 con 0x7f04180b6110 2023-07-06T20:13:43.058 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f04177fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 0x7f041815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:43.058 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f04177fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 0x7f041815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55960/0 (socket says 172.21.15.27:55960) 2023-07-06T20:13:43.058 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f04177fe700 1 -- 172.21.15.27:0/908618826 learned_addr learned my addr 172.21.15.27:0/908618826 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:43.058 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f0417fff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f041815fca0 0x7f04181f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:43.058 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f0416ffd700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04180b6110 0x7f04181f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:43.059 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f0417fff700 1 -- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04180b6110 msgr2=0x7f04181f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.059 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.055+0000 7f0417fff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04180b6110 0x7f04181f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f0417fff700 1 -- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 msgr2=0x7f041815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f0417fff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 0x7f041815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f0417fff700 1 -- 172.21.15.27:0/908618826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0094a0 con 0x7f041815fca0 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f04177fe700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 0x7f041815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f0417fff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f041815fca0 0x7f04181f51a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f041000cb90 tx=0x7f041000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410003c00 con 0x7f041815fca0 2023-07-06T20:13:43.060 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410003d60 con 0x7f041815fca0 2023-07-06T20:13:43.061 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000aa80 con 0x7f041815fca0 2023-07-06T20:13:43.061 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04181f5df0 con 0x7f041815fca0 2023-07-06T20:13:43.061 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.056+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04181f2030 con 0x7f041815fca0 2023-07-06T20:13:43.061 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.057+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04180008d0 con 0x7f041815fca0 2023-07-06T20:13:43.062 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.058+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f041000abe0 con 0x7f041815fca0 2023-07-06T20:13:43.062 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.059+0000 7f03fffff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f040406ac50 0x7f040406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:43.062 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.059+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f04100882f0 con 0x7f041815fca0 2023-07-06T20:13:43.062 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.060+0000 7f04177fe700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f040406ac50 0x7f040406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:43.062 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.060+0000 7f04177fe700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f040406ac50 0x7f040406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0408007cd0 tx=0x7f0408007c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:43.066 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.064+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0410054400 con 0x7f041815fca0 2023-07-06T20:13:43.125 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-07-06T20:13:43.124+0000 7ff6343f6700 -1 osd.7 0 waiting for initial osdmap 2023-07-06T20:13:43.137 INFO:tasks.ceph.osd.7.smithi153.stderr:2023-07-06T20:13:43.135+0000 7ff62c3fb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:43.276 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.274+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04180b2c40 con 0x7f041815fca0 2023-07-06T20:13:43.276 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.275+0000 7f03fffff700 1 -- 172.21.15.27:0/908618826 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0410057a20 con 0x7f041815fca0 2023-07-06T20:13:43.277 INFO:teuthology.misc.health.smithi027.stdout: 2023-07-06T20:13:43.277 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":12,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:40.382450+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:13:43.279 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f040406ac50 msgr2=0x7f040406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.279 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f040406ac50 0x7f040406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0408007cd0 tx=0x7f0408007c20 comp rx=0 tx=0).stop 2023-07-06T20:13:43.279 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f041815fca0 msgr2=0x7f04181f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f041815fca0 0x7f04181f51a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f041000cb90 tx=0x7f041000cea0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 shutdown_connections 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f040406ac50 0x7f040406d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f041815fca0 0x7f04181f51a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04180b5c90 0x7f041815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 --2- 172.21.15.27:0/908618826 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04180b6110 0x7f04181f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:43.280 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.277+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 >> 172.21.15.27:0/908618826 conn(0x7f041806c150 msgr2=0x7f04180d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:43.281 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.278+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 shutdown_connections 2023-07-06T20:13:43.281 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:43.278+0000 7f041e1ff700 1 -- 172.21.15.27:0/908618826 wait complete. 2023-07-06T20:13:43.295 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T20:13:50.099 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T20:13:50.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:13:50.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:13:50.295 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.293+0000 7fe9373ff700 1 Processor -- start 2023-07-06T20:13:50.295 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.293+0000 7fe9373ff700 1 -- start start 2023-07-06T20:13:50.296 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe9373ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.297 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe9373ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.297 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe9373ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9300cf780 con 0x7fe9300b6100 2023-07-06T20:13:50.297 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe9373ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9300cd490 con 0x7fe9300b5c90 2023-07-06T20:13:50.297 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe9373ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9300cd770 con 0x7fe93015fca0 2023-07-06T20:13:50.297 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe93599c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9300b6100 0x7fe9300cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43444/0 (socket says 172.21.15.27:43444) 2023-07-06T20:13:50.297 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe93599c700 1 -- 172.21.15.27:0/729095013 learned_addr learned my addr 172.21.15.27:0/729095013 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:50.299 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe93519b700 1 --2- 172.21.15.27:0/729095013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.299 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe93499a700 1 --2- 172.21.15.27:0/729095013 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.299 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.295+0000 7fe927fff700 1 -- 172.21.15.27:0/729095013 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973750977 0 0) 0x7fe9300cf780 con 0x7fe9300b6100 2023-07-06T20:13:50.299 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe927fff700 1 -- 172.21.15.27:0/729095013 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe91c0035b0 con 0x7fe9300b6100 2023-07-06T20:13:50.299 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe93519b700 1 -- 172.21.15.27:0/729095013 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 msgr2=0x7fe9300d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:50.299 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe93519b700 1 --2- 172.21.15.27:0/729095013 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe93519b700 1 -- 172.21.15.27:0/729095013 >> v1:172.21.15.27:6789/0 conn(0x7fe9300b6100 legacy=0x7fe9300cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe93519b700 1 -- 172.21.15.27:0/729095013 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9300cda50 con 0x7fe93015fca0 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe93499a700 1 --2- 172.21.15.27:0/729095013 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe93519b700 1 --2- 172.21.15.27:0/729095013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300d1420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009840 tx=0x7fe92c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e420f116ccfa66e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe927fff700 1 -- 172.21.15.27:0/729095013 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c011830 con 0x7fe93015fca0 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe927fff700 1 -- 172.21.15.27:0/729095013 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c011990 con 0x7fe93015fca0 2023-07-06T20:13:50.300 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.296+0000 7fe927fff700 1 -- 172.21.15.27:0/729095013 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c011d20 con 0x7fe93015fca0 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 -- 172.21.15.27:0/729095013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 msgr2=0x7fe9300d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 --2- 172.21.15.27:0/729095013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300d1420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009840 tx=0x7fe92c009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 -- 172.21.15.27:0/729095013 shutdown_connections 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 --2- 172.21.15.27:0/729095013 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 --2- 172.21.15.27:0/729095013 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 -- 172.21.15.27:0/729095013 >> 172.21.15.27:0/729095013 conn(0x7fe93006c150 msgr2=0x7fe93006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 -- 172.21.15.27:0/729095013 shutdown_connections 2023-07-06T20:13:50.301 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.297+0000 7fe9373ff700 1 -- 172.21.15.27:0/729095013 wait complete. 2023-07-06T20:13:50.303 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.299+0000 7fe9373ff700 1 Processor -- start 2023-07-06T20:13:50.303 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.299+0000 7fe9373ff700 1 -- start start 2023-07-06T20:13:50.303 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe9373ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 0x7fe9300b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.303 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe9373ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b6100 0x7fe9300cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.303 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe9373ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe9373ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93014a720 con 0x7fe9300b5c90 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe9373ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9300ca0b0 con 0x7fe9300b6100 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe9373ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9300cc6d0 con 0x7fe93015fca0 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe93519b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 0x7fe9300b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe93519b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 0x7fe9300b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49978/0 (socket says 172.21.15.27:49978) 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe93519b700 1 -- 172.21.15.27:0/1401455822 learned_addr learned my addr 172.21.15.27:0/1401455822 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:50.304 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe93599c700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.300+0000 7fe93499a700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b6100 0x7fe9300cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe93519b700 1 -- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 msgr2=0x7fe9300c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe93519b700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe93519b700 1 -- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b6100 msgr2=0x7fe9300cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe93519b700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b6100 0x7fe9300cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe93519b700 1 -- 172.21.15.27:0/1401455822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9200094a0 con 0x7fe9300b5c90 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe93519b700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 0x7fe9300b32d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe92c014040 tx=0x7fe92c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:50.305 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c004590 con 0x7fe9300b5c90 2023-07-06T20:13:50.306 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c01f070 con 0x7fe9300b5c90 2023-07-06T20:13:50.306 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0094a0 con 0x7fe9300b5c90 2023-07-06T20:13:50.306 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0038a0 con 0x7fe9300b5c90 2023-07-06T20:13:50.306 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.301+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9300b3b90 con 0x7fe9300b5c90 2023-07-06T20:13:50.307 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.302+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9300008d0 con 0x7fe9300b5c90 2023-07-06T20:13:50.307 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.304+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe92c003a00 con 0x7fe9300b5c90 2023-07-06T20:13:50.307 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.304+0000 7fe925ffb700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91406ad30 0x7fe91406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.307 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.305+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe92c021070 con 0x7fe9300b5c90 2023-07-06T20:13:50.307 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.305+0000 7fe93499a700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91406ad30 0x7fe91406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.308 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.306+0000 7fe93499a700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91406ad30 0x7fe91406d1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9301f6760 tx=0x7fe920009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:50.311 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.310+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe92c055c80 con 0x7fe9300b5c90 2023-07-06T20:13:50.521 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.519+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9300c97a0 con 0x7fe9300b5c90 2023-07-06T20:13:50.522 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.520+0000 7fe925ffb700 1 -- 172.21.15.27:0/1401455822 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12826 (secure 0 0 0) 0x7fe92c003cb0 con 0x7fe9300b5c90 2023-07-06T20:13:50.522 INFO:teuthology.misc.health.smithi027.stdout: 2023-07-06T20:13:50.523 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":16,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:46.537695+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:13:50.525 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91406ad30 msgr2=0x7fe91406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.525 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91406ad30 0x7fe91406d1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9301f6760 tx=0x7fe920009210 comp rx=0 tx=0).stop 2023-07-06T20:13:50.525 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 msgr2=0x7fe9300b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.525 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 0x7fe9300b32d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe92c014040 tx=0x7fe92c004750 comp rx=0 tx=0).stop 2023-07-06T20:13:50.525 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 shutdown_connections 2023-07-06T20:13:50.525 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91406ad30 0x7fe91406d1e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.526 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe93015fca0 0x7fe9300c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.526 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9300b5c90 0x7fe9300b32d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.526 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 --2- 172.21.15.27:0/1401455822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b6100 0x7fe9300cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.526 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 >> 172.21.15.27:0/1401455822 conn(0x7fe93006c150 msgr2=0x7fe9300b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:50.526 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 shutdown_connections 2023-07-06T20:13:50.526 INFO:teuthology.misc.health.smithi027.stderr:2023-07-06T20:13:50.523+0000 7fe9373ff700 1 -- 172.21.15.27:0/1401455822 wait complete. 2023-07-06T20:13:50.540 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-06T20:13:50.541 INFO:tasks.ceph:Creating RBD pool 2023-07-06T20:13:50.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-06T20:13:50.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.737+0000 7f496f3ff700 1 Processor -- start 2023-07-06T20:13:50.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.737+0000 7f496f3ff700 1 -- start start 2023-07-06T20:13:50.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496f3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b6110 0x7f49680cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496f3ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49680cf780 con 0x7f496815fca0 2023-07-06T20:13:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496f3ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49680cd490 con 0x7f49680b5c90 2023-07-06T20:13:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496f3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49680cd770 con 0x7f49680b6110 2023-07-06T20:13:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496d19b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f496815fca0 0x7f49680d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43456/0 (socket says 172.21.15.27:43456) 2023-07-06T20:13:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496d19b700 1 -- 172.21.15.27:0/1218292450 learned_addr learned my addr 172.21.15.27:0/1218292450 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496d99c700 1 --2- 172.21.15.27:0/1218292450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b6110 0x7f49680cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466143164 0 0) 0x7f49680cf780 con 0x7f496815fca0 2023-07-06T20:13:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49540035b0 con 0x7f496815fca0 2023-07-06T20:13:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723069539 0 0) 0x7f49680cd490 con 0x7f49680b5c90 2023-07-06T20:13:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49680cf780 con 0x7f49680b5c90 2023-07-06T20:13:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496d99c700 1 -- 172.21.15.27:0/1218292450 >> v1:172.21.15.153:6789/0 conn(0x7f49680b5c90 legacy=0x7f49680d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.739+0000 7f496d99c700 1 -- 172.21.15.27:0/1218292450 >> v1:172.21.15.27:6789/0 conn(0x7f496815fca0 legacy=0x7f49680d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.740+0000 7f496d99c700 1 -- 172.21.15.27:0/1218292450 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680cda50 con 0x7f49680b6110 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.740+0000 7f496d99c700 1 --2- 172.21.15.27:0/1218292450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b6110 0x7f49680cfc10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4960009dc0 tx=0x7f496000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5decfa1d6ab56b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.740+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000e070 con 0x7f49680b6110 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.740+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496000bb20 con 0x7f49680b6110 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.740+0000 7f495ffff700 1 -- 172.21.15.27:0/1218292450 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000be90 con 0x7f49680b6110 2023-07-06T20:13:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 -- 172.21.15.27:0/1218292450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b6110 msgr2=0x7f49680cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1218292450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b6110 0x7f49680cfc10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4960009dc0 tx=0x7f496000b330 comp rx=0 tx=0).stop 2023-07-06T20:13:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 -- 172.21.15.27:0/1218292450 shutdown_connections 2023-07-06T20:13:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1218292450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b6110 0x7f49680cfc10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 -- 172.21.15.27:0/1218292450 >> 172.21.15.27:0/1218292450 conn(0x7f496806c150 msgr2=0x7f496806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 -- 172.21.15.27:0/1218292450 shutdown_connections 2023-07-06T20:13:50.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.741+0000 7f496f3ff700 1 -- 172.21.15.27:0/1218292450 wait complete. 2023-07-06T20:13:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.743+0000 7f496f3ff700 1 Processor -- start 2023-07-06T20:13:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.744+0000 7f496f3ff700 1 -- start start 2023-07-06T20:13:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496f3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b5c90 0x7f496814c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496f3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49680b6110 0x7f496815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496f3ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 0x7f496815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496f3ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49681f5150 con 0x7f496815fca0 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496f3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680cbc80 con 0x7f49680b6110 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496f3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49680cbf60 con 0x7f49680b5c90 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 0x7f496815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 0x7f496815cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49980/0 (socket says 172.21.15.27:49980) 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 -- 172.21.15.27:0/1356247381 learned_addr learned my addr 172.21.15.27:0/1356247381 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d19b700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b5c90 0x7f496814c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496c99a700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49680b6110 0x7f496815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 -- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b5c90 msgr2=0x7f496814c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b5c90 0x7f496814c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 -- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49680b6110 msgr2=0x7f496815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49680b6110 0x7f496815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.745+0000 7f496d99c700 1 -- 172.21.15.27:0/1356247381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4960009a70 con 0x7f496815fca0 2023-07-06T20:13:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.746+0000 7f496d99c700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 0x7f496815cc30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4960014040 tx=0x7f4960004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.746+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49600098f0 con 0x7f496815fca0 2023-07-06T20:13:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.746+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49680cc240 con 0x7f496815fca0 2023-07-06T20:13:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.746+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960020070 con 0x7f496815fca0 2023-07-06T20:13:50.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.746+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49680cc520 con 0x7f496815fca0 2023-07-06T20:13:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.746+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960027680 con 0x7f496815fca0 2023-07-06T20:13:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.747+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49681f5320 con 0x7f496815fca0 2023-07-06T20:13:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.748+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49600277e0 con 0x7f496815fca0 2023-07-06T20:13:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.749+0000 7f495dffb700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f494c06ace0 0x7f494c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.749+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4960022070 con 0x7f496815fca0 2023-07-06T20:13:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.750+0000 7f496d19b700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f494c06ace0 0x7f494c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.750+0000 7f496d19b700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f494c06ace0 0x7f494c06d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4964000f60 tx=0x7f4964009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:50.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.754+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49600566d0 con 0x7f496815fca0 2023-07-06T20:13:50.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:50.965+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f49680008d0 con 0x7f496815fca0 2023-07-06T20:13:51.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.961+0000 7f495dffb700 1 -- 172.21.15.27:0/1356247381 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f4960059cf0 con 0x7f496815fca0 2023-07-06T20:13:51.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f494c06ace0 msgr2=0x7f494c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f494c06ace0 0x7f494c06d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4964000f60 tx=0x7f4964009040 comp rx=0 tx=0).stop 2023-07-06T20:13:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 msgr2=0x7f496815cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 0x7f496815cc30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4960014040 tx=0x7f4960004ad0 comp rx=0 tx=0).stop 2023-07-06T20:13:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 shutdown_connections 2023-07-06T20:13:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f494c06ace0 0x7f494c06d190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f496815fca0 0x7f496815cc30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49680b5c90 0x7f496814c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 --2- 172.21.15.27:0/1356247381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49680b6110 0x7f496815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.966+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 >> 172.21.15.27:0/1356247381 conn(0x7f496806c150 msgr2=0x7f49680b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.967+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 shutdown_connections 2023-07-06T20:13:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:51.967+0000 7f496f3ff700 1 -- 172.21.15.27:0/1356247381 wait complete. 2023-07-06T20:13:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'rbd' created 2023-07-06T20:13:51.990 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-06T20:13:52.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.193+0000 7f69d7fff700 1 Processor -- start 2023-07-06T20:13:52.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.193+0000 7f69d7fff700 1 -- start start 2023-07-06T20:13:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.194+0000 7f69d7fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69d00cfe40 0x7f69d00cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.194+0000 7f69d7fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d0160cc0 con 0x7f69d00b2180 2023-07-06T20:13:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.194+0000 7f69d7fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d0160f80 con 0x7f69d00b8440 2023-07-06T20:13:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.194+0000 7f69d7fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d0160ae0 con 0x7f69d00cfe40 2023-07-06T20:13:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d559a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69d00cfe40 0x7f69d00cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d659c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f69d00b2180 0x7f69d00d0960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46014/0 (socket says 172.21.15.27:46014) 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d559a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69d00cfe40 0x7f69d00cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46440/0 (socket says 172.21.15.27:46440) 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d659c700 1 -- 172.21.15.27:0/2085267144 learned_addr learned my addr 172.21.15.27:0/2085267144 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d559a700 1 -- 172.21.15.27:0/2085267144 >> v1:172.21.15.153:6789/0 conn(0x7f69d00b8440 legacy=0x7f69d00cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d559a700 1 -- 172.21.15.27:0/2085267144 >> v1:172.21.15.27:6789/0 conn(0x7f69d00b2180 legacy=0x7f69d00d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d4d99700 1 -- 172.21.15.27:0/2085267144 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812889445 0 0) 0x7f69d0160f80 con 0x7f69d00b8440 2023-07-06T20:13:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d559a700 1 -- 172.21.15.27:0/2085267144 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00cf0e0 con 0x7f69d00cfe40 2023-07-06T20:13:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.195+0000 7f69d559a700 1 --2- 172.21.15.27:0/2085267144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00cfe40 0x7f69d00cd530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009840 tx=0x7f69c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c84aa5aa1883b66 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d4d99700 1 -- 172.21.15.27:0/2085267144 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0004230 con 0x7f69d00cfe40 2023-07-06T20:13:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d4d99700 1 -- 172.21.15.27:0/2085267144 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0004390 con 0x7f69d00cfe40 2023-07-06T20:13:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d4d99700 1 -- 172.21.15.27:0/2085267144 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c00036a0 con 0x7f69d00cfe40 2023-07-06T20:13:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d7fff700 1 -- 172.21.15.27:0/2085267144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00cfe40 msgr2=0x7f69d00cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d7fff700 1 --2- 172.21.15.27:0/2085267144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00cfe40 0x7f69d00cd530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009840 tx=0x7f69c0009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d7fff700 1 -- 172.21.15.27:0/2085267144 shutdown_connections 2023-07-06T20:13:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d7fff700 1 --2- 172.21.15.27:0/2085267144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00cfe40 0x7f69d00cd530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d7fff700 1 -- 172.21.15.27:0/2085267144 >> 172.21.15.27:0/2085267144 conn(0x7f69d006be10 msgr2=0x7f69d0071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:52.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.196+0000 7f69d7fff700 1 -- 172.21.15.27:0/2085267144 shutdown_connections 2023-07-06T20:13:52.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.197+0000 7f69d7fff700 1 -- 172.21.15.27:0/2085267144 wait complete. 2023-07-06T20:13:52.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.198+0000 7f69d7fff700 1 Processor -- start 2023-07-06T20:13:52.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.198+0000 7f69d7fff700 1 -- start start 2023-07-06T20:13:52.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.199+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d00b2180 0x7f69d015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:52.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.199+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 0x7f69d01e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:52.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.199+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69d00cfe40 0x7f69d01e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:52.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.199+0000 7f69d7fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d01568c0 con 0x7f69d00b8440 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.199+0000 7f69d7fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d015e250 con 0x7f69d00b2180 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.199+0000 7f69d7fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d015e530 con 0x7f69d00cfe40 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 0x7f69d01e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 0x7f69d01e5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46442/0 (socket says 172.21.15.27:46442) 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 -- 172.21.15.27:0/1393252019 learned_addr learned my addr 172.21.15.27:0/1393252019 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d659c700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69d00cfe40 0x7f69d01e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:52.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d5d9b700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d00b2180 0x7f69d015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:52.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 -- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69d00cfe40 msgr2=0x7f69d01e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69d00cfe40 0x7f69d01e63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 -- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d00b2180 msgr2=0x7f69d015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d00b2180 0x7f69d015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 -- 172.21.15.27:0/1393252019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c00094a0 con 0x7f69d00b8440 2023-07-06T20:13:52.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.200+0000 7f69d559a700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 0x7f69d01e5ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69c00090b0 tx=0x7f69c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:52.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.201+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0003e70 con 0x7f69d00b8440 2023-07-06T20:13:52.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.201+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c001f070 con 0x7f69d00b8440 2023-07-06T20:13:52.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.201+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0011850 con 0x7f69d00b8440 2023-07-06T20:13:52.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.201+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d015e810 con 0x7f69d00b8440 2023-07-06T20:13:52.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.201+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d015edd0 con 0x7f69d00b8440 2023-07-06T20:13:52.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.202+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d015ef60 con 0x7f69d00b8440 2023-07-06T20:13:52.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.203+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69c0021030 con 0x7f69d00b8440 2023-07-06T20:13:52.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.204+0000 7f69c67fc700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f69b406ace0 0x7f69b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:52.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.204+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f69c008aa40 con 0x7f69d00b8440 2023-07-06T20:13:52.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.205+0000 7f69d5d9b700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f69b406ace0 0x7f69b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:52.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.205+0000 7f69d5d9b700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f69b406ace0 0x7f69b406d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000f60 tx=0x7f69cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:52.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.209+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f69c0055fc0 con 0x7f69d00b8440 2023-07-06T20:13:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.417+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f69d015ef60 con 0x7f69d00b8440 2023-07-06T20:13:52.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.969+0000 7f69c67fc700 1 -- 172.21.15.27:0/1393252019 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f69c00595e0 con 0x7f69d00b8440 2023-07-06T20:13:52.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f69b406ace0 msgr2=0x7f69b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f69b406ace0 0x7f69b406d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000f60 tx=0x7f69cc009040 comp rx=0 tx=0).stop 2023-07-06T20:13:52.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 msgr2=0x7f69d01e5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:52.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 0x7f69d01e5ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69c00090b0 tx=0x7f69c0004750 comp rx=0 tx=0).stop 2023-07-06T20:13:52.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 shutdown_connections 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f69b406ace0 0x7f69b406d190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69d00cfe40 0x7f69d01e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d00b2180 0x7f69d015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 --2- 172.21.15.27:0/1393252019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69d00b8440 0x7f69d01e5ea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.974+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 >> 172.21.15.27:0/1393252019 conn(0x7f69d006be10 msgr2=0x7f69d00b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.975+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 shutdown_connections 2023-07-06T20:13:52.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:52.975+0000 7f69d7fff700 1 -- 172.21.15.27:0/1393252019 wait complete. 2023-07-06T20:13:52.979 INFO:teuthology.orchestra.run.smithi027.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-06T20:13:52.999 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-06T20:13:52.999 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T20:13:53.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T20:13:53.004 INFO:tasks.ceph.mds.a:Started 2023-07-06T20:13:53.004 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T20:13:53.004 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T20:13:53.008 INFO:tasks.ceph.mds.c:Started 2023-07-06T20:13:53.008 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T20:13:53.009 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T20:13:53.013 INFO:tasks.ceph.mds.b:Started 2023-07-06T20:13:53.014 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T20:13:53.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T20:13:53.018 INFO:tasks.ceph.mds.d:Started 2023-07-06T20:13:53.018 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-06T20:13:53.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:13:53.164 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-07-06T20:13:53.165 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-07-06T20:13:53.165 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-07-06T20:13:53.165 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-07-06T20:13:53.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.232+0000 7f8883dff700 1 Processor -- start 2023-07-06T20:13:53.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.232+0000 7f8883dff700 1 -- start start 2023-07-06T20:13:53.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.233+0000 7f8883dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f887c0c9010 0x7f887c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.233+0000 7f8883dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f887c1565f0 0x7f887c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f8883dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0c9a60 con 0x7f887c06c330 2023-07-06T20:13:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f8883dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c159b70 con 0x7f887c0c9010 2023-07-06T20:13:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f8883dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c159e50 con 0x7f887c1565f0 2023-07-06T20:13:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f887c0c9010 0x7f887c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f887c0c9010 0x7f887c0c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46478/0 (socket says 172.21.15.27:46478) 2023-07-06T20:13:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 -- 172.21.15.27:0/3102391317 learned_addr learned my addr 172.21.15.27:0/3102391317 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:53.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888239c700 1 --2- 172.21.15.27:0/3102391317 >> v2:172.21.15.153:3300/0 conn(0x7f887c1565f0 0x7f887c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 -- 172.21.15.27:0/3102391317 >> v2:172.21.15.153:3300/0 conn(0x7f887c1565f0 msgr2=0x7f887c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 --2- 172.21.15.27:0/3102391317 >> v2:172.21.15.153:3300/0 conn(0x7f887c1565f0 0x7f887c156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 -- 172.21.15.27:0/3102391317 >> v1:172.21.15.27:6789/0 conn(0x7f887c06c330 legacy=0x7f887c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 -- 172.21.15.27:0/3102391317 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c15a130 con 0x7f887c0c9010 2023-07-06T20:13:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888239c700 1 --2- 172.21.15.27:0/3102391317 >> v2:172.21.15.153:3300/0 conn(0x7f887c1565f0 0x7f887c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:13:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.234+0000 7f888139a700 1 --2- 172.21.15.27:0/3102391317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c0c9010 0x7f887c0c9480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f887c06f6e0 tx=0x7f886c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a86c597a247db6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8880b99700 1 -- 172.21.15.27:0/3102391317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00b870 con 0x7f887c0c9010 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8880b99700 1 -- 172.21.15.27:0/3102391317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c00b9d0 con 0x7f887c0c9010 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8880b99700 1 -- 172.21.15.27:0/3102391317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00bd60 con 0x7f887c0c9010 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 -- 172.21.15.27:0/3102391317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c0c9010 msgr2=0x7f887c0c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 --2- 172.21.15.27:0/3102391317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c0c9010 0x7f887c0c9480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f887c06f6e0 tx=0x7f886c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 -- 172.21.15.27:0/3102391317 shutdown_connections 2023-07-06T20:13:53.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 --2- 172.21.15.27:0/3102391317 >> v2:172.21.15.153:3300/0 conn(0x7f887c1565f0 0x7f887c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 --2- 172.21.15.27:0/3102391317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c0c9010 0x7f887c0c9480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 -- 172.21.15.27:0/3102391317 >> 172.21.15.27:0/3102391317 conn(0x7f887c0cb440 msgr2=0x7f887c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:53.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 -- 172.21.15.27:0/3102391317 shutdown_connections 2023-07-06T20:13:53.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.235+0000 7f8883dff700 1 -- 172.21.15.27:0/3102391317 wait complete. 2023-07-06T20:13:53.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.237+0000 7f8883dff700 1 Processor -- start 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.237+0000 7f8883dff700 1 -- start start 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8883dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 0x7f887c1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8883dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f887c0c9010 0x7f887c0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8883dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f887c1565f0 0x7f887c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8883dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b2d70 con 0x7f887c06c330 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8883dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b2160 con 0x7f887c0c9010 2023-07-06T20:13:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8883dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b2440 con 0x7f887c1565f0 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.238+0000 7f8881b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 0x7f887c1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 0x7f887c1556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46494/0 (socket says 172.21.15.27:46494) 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 -- 172.21.15.27:0/3509927122 learned_addr learned my addr 172.21.15.27:0/3509927122 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f888239c700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f887c1565f0 0x7f887c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f888139a700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f887c0c9010 0x7f887c0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 -- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f887c1565f0 msgr2=0x7f887c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f887c1565f0 0x7f887c0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 -- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f887c0c9010 msgr2=0x7f887c0b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f887c0c9010 0x7f887c0b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 -- 172.21.15.27:0/3509927122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c009550 con 0x7f887c06c330 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f888239c700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f887c1565f0 0x7f887c0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f888139a700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f887c0c9010 0x7f887c0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8881b9b700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 0x7f887c1556a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f887800c450 tx=0x7f887800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780052d0 con 0x7f887c06c330 2023-07-06T20:13:53.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878005430 con 0x7f887c06c330 2023-07-06T20:13:53.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c0b2720 con 0x7f887c06c330 2023-07-06T20:13:53.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.239+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780038c0 con 0x7f887c06c330 2023-07-06T20:13:53.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.240+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c0b3200 con 0x7f887c06c330 2023-07-06T20:13:53.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.240+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c0008d0 con 0x7f887c06c330 2023-07-06T20:13:53.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.242+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8878003a20 con 0x7f887c06c330 2023-07-06T20:13:53.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.243+0000 7f88727fc700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f886006ace0 0x7f886006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.243+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f88780886e0 con 0x7f887c06c330 2023-07-06T20:13:53.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.243+0000 7f888139a700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f886006ace0 0x7f886006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.244+0000 7f888139a700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f886006ace0 0x7f886006d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f887c06f6e0 tx=0x7f886c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:53.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.246+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8878053ac0 con 0x7f887c06c330 2023-07-06T20:13:53.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.456+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f887c0cc2d0 con 0x7f887c06c330 2023-07-06T20:13:53.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.457+0000 7f88727fc700 1 -- 172.21.15.27:0/3509927122 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14625 (secure 0 0 0) 0x7f88780570e0 con 0x7f887c06c330 2023-07-06T20:13:53.460 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:13:53.461 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:52.962144+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:13:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f886006ace0 msgr2=0x7f886006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f886006ace0 0x7f886006d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f887c06f6e0 tx=0x7f886c008030 comp rx=0 tx=0).stop 2023-07-06T20:13:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 msgr2=0x7f887c1556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 0x7f887c1556a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f887800c450 tx=0x7f887800c810 comp rx=0 tx=0).stop 2023-07-06T20:13:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 shutdown_connections 2023-07-06T20:13:53.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f886006ace0 0x7f886006d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f887c1565f0 0x7f887c0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f887c06c330 0x7f887c1556a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 --2- 172.21.15.27:0/3509927122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f887c0c9010 0x7f887c0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 >> 172.21.15.27:0/3509927122 conn(0x7f887c0cb440 msgr2=0x7f887c15a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 shutdown_connections 2023-07-06T20:13:53.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.460+0000 7f8883dff700 1 -- 172.21.15.27:0/3509927122 wait complete. 2023-07-06T20:13:53.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:13:53.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:13:53.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.684+0000 7fc1755ff700 1 Processor -- start 2023-07-06T20:13:53.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.684+0000 7fc1755ff700 1 -- start start 2023-07-06T20:13:53.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc1755ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17015fca0 0x7fc1700d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc1755ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1700b5c90 0x7fc1700d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc1755ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1700b6100 0x7fc1700d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc1755ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1700cf780 con 0x7fc1700b5c90 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc1755ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1700cd490 con 0x7fc1700b6100 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc1755ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700cd770 con 0x7fc17015fca0 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc16e7fc700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1700b5c90 0x7fc1700d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc16e7fc700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1700b5c90 0x7fc1700d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46504/0 (socket says 172.21.15.27:46504) 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc16f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1700b6100 0x7fc1700d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc16e7fc700 1 -- 172.21.15.27:0/3800448496 learned_addr learned my addr 172.21.15.27:0/3800448496 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:53.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.685+0000 7fc16effd700 1 --2- 172.21.15.27:0/3800448496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17015fca0 0x7fc1700d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16e7fc700 1 -- 172.21.15.27:0/3800448496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17015fca0 msgr2=0x7fc1700d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16e7fc700 1 --2- 172.21.15.27:0/3800448496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17015fca0 0x7fc1700d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16e7fc700 1 -- 172.21.15.27:0/3800448496 >> v2:172.21.15.153:3300/0 conn(0x7fc1700b6100 msgr2=0x7fc1700d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16e7fc700 1 --2- 172.21.15.27:0/3800448496 >> v2:172.21.15.153:3300/0 conn(0x7fc1700b6100 0x7fc1700d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16e7fc700 1 -- 172.21.15.27:0/3800448496 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700cda50 con 0x7fc1700b5c90 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16e7fc700 1 --2- 172.21.15.27:0/3800448496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1700b5c90 0x7fc1700d2d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1700b8f80 tx=0x7fc164009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad5310319b7c259 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16dffb700 1 -- 172.21.15.27:0/3800448496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164011830 con 0x7fc1700b5c90 2023-07-06T20:13:53.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16dffb700 1 -- 172.21.15.27:0/3800448496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc164011990 con 0x7fc1700b5c90 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.686+0000 7fc16dffb700 1 -- 172.21.15.27:0/3800448496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164011d20 con 0x7fc1700b5c90 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 -- 172.21.15.27:0/3800448496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1700b5c90 msgr2=0x7fc1700d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 --2- 172.21.15.27:0/3800448496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1700b5c90 0x7fc1700d2d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1700b8f80 tx=0x7fc164009bd0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 -- 172.21.15.27:0/3800448496 shutdown_connections 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 --2- 172.21.15.27:0/3800448496 >> v2:172.21.15.153:3300/0 conn(0x7fc1700b6100 0x7fc1700d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 --2- 172.21.15.27:0/3800448496 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc17015fca0 0x7fc1700d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 --2- 172.21.15.27:0/3800448496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1700b5c90 0x7fc1700d2d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 -- 172.21.15.27:0/3800448496 >> 172.21.15.27:0/3800448496 conn(0x7fc17006c150 msgr2=0x7fc17006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:53.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 -- 172.21.15.27:0/3800448496 shutdown_connections 2023-07-06T20:13:53.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.687+0000 7fc1755ff700 1 -- 172.21.15.27:0/3800448496 wait complete. 2023-07-06T20:13:53.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.689+0000 7fc1755ff700 1 Processor -- start 2023-07-06T20:13:53.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.689+0000 7fc1755ff700 1 -- start start 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc1755ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1700b5c90 0x7fc17015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc1755ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1700b6100 0x7fc1700b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc1755ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 0x7fc170146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc1755ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170148a50 con 0x7fc17015fca0 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc1755ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170148d30 con 0x7fc1700b6100 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc1755ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc170149010 con 0x7fc1700b5c90 2023-07-06T20:13:53.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc16f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 0x7fc170146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc16f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 0x7fc170146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46510/0 (socket says 172.21.15.27:46510) 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.690+0000 7fc16f7fe700 1 -- 172.21.15.27:0/2659541648 learned_addr learned my addr 172.21.15.27:0/2659541648 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16effd700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1700b5c90 0x7fc17015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16e7fc700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1700b6100 0x7fc1700b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16f7fe700 1 -- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1700b5c90 msgr2=0x7fc17015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16f7fe700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1700b5c90 0x7fc17015d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16f7fe700 1 -- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1700b6100 msgr2=0x7fc1700b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16f7fe700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1700b6100 0x7fc1700b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16f7fe700 1 -- 172.21.15.27:0/2659541648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009550 con 0x7fc17015fca0 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc16f7fe700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 0x7fc170146400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc16800eed0 tx=0x7fc168009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168010040 con 0x7fc17015fca0 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16800b9e0 con 0x7fc17015fca0 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168009b20 con 0x7fc17015fca0 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.691+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1701e5f80 con 0x7fc17015fca0 2023-07-06T20:13:53.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.692+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701492f0 con 0x7fc17015fca0 2023-07-06T20:13:53.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.693+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1701e61c0 con 0x7fc17015fca0 2023-07-06T20:13:53.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.694+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc16801d400 con 0x7fc17015fca0 2023-07-06T20:13:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.695+0000 7fc1577fe700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc15c06ace0 0x7fc15c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.695+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc168089780 con 0x7fc17015fca0 2023-07-06T20:13:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.695+0000 7fc16effd700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc15c06ace0 0x7fc15c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:53.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.695+0000 7fc16effd700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc15c06ace0 0x7fc15c06d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc160009be0 tx=0x7fc160009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:53.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.699+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc1701e61c0 con 0x7fc17015fca0 2023-07-06T20:13:53.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.905+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1701e61c0 con 0x7fc17015fca0 2023-07-06T20:13:53.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.906+0000 7fc1577fe700 1 -- 172.21.15.27:0/2659541648 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc168018020 con 0x7fc17015fca0 2023-07-06T20:13:53.909 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:13:53.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc15c06ace0 msgr2=0x7fc15c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc15c06ace0 0x7fc15c06d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc160009be0 tx=0x7fc160009210 comp rx=0 tx=0).stop 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 msgr2=0x7fc170146400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 0x7fc170146400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc16800eed0 tx=0x7fc168009f60 comp rx=0 tx=0).stop 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 shutdown_connections 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc15c06ace0 0x7fc15c06d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc17015fca0 0x7fc170146400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1700b5c90 0x7fc17015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 --2- 172.21.15.27:0/2659541648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1700b6100 0x7fc1700b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:53.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 >> 172.21.15.27:0/2659541648 conn(0x7fc17006c150 msgr2=0x7fc1700b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:53.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 shutdown_connections 2023-07-06T20:13:53.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:53.909+0000 7fc1755ff700 1 -- 172.21.15.27:0/2659541648 wait complete. 2023-07-06T20:13:53.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:13:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.135+0000 7fb9153ff700 1 Processor -- start 2023-07-06T20:13:54.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.135+0000 7fb9153ff700 1 -- start start 2023-07-06T20:13:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb9153ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb9100d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb9153ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb9100b5c90 0x7fb9100d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb9153ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9100b6100 0x7fb9100d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb9153ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9100cf780 con 0x7fb9100b6100 2023-07-06T20:13:54.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb9153ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9100cd490 con 0x7fb9100b5c90 2023-07-06T20:13:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb9153ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100cd770 con 0x7fb91015fca0 2023-07-06T20:13:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb90f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9100b6100 0x7fb9100d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb90f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9100b6100 0x7fb9100d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46512/0 (socket says 172.21.15.27:46512) 2023-07-06T20:13:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb90f7fe700 1 -- 172.21.15.27:0/2074443787 learned_addr learned my addr 172.21.15.27:0/2074443787 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb90effd700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb9100d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.136+0000 7fb90e7fc700 1 --2- 172.21.15.27:0/2074443787 >> v2:172.21.15.153:3300/0 conn(0x7fb9100b5c90 0x7fb9100d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90f7fe700 1 -- 172.21.15.27:0/2074443787 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 msgr2=0x7fb9100d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90f7fe700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb9100d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90f7fe700 1 -- 172.21.15.27:0/2074443787 >> v2:172.21.15.153:3300/0 conn(0x7fb9100b5c90 msgr2=0x7fb9100d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90f7fe700 1 --2- 172.21.15.27:0/2074443787 >> v2:172.21.15.153:3300/0 conn(0x7fb9100b5c90 0x7fb9100d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90f7fe700 1 -- 172.21.15.27:0/2074443787 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100cda50 con 0x7fb9100b6100 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90effd700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb9100d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:13:54.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90f7fe700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b6100 0x7fb9100d3280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb91006cda0 tx=0x7fb90800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b3b447e735ba55 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90dffb700 1 -- 172.21.15.27:0/2074443787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90800e070 con 0x7fb9100b6100 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90dffb700 1 -- 172.21.15.27:0/2074443787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90800b910 con 0x7fb9100b6100 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.137+0000 7fb90dffb700 1 -- 172.21.15.27:0/2074443787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90800bcc0 con 0x7fb9100b6100 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 -- 172.21.15.27:0/2074443787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b6100 msgr2=0x7fb9100d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b6100 0x7fb9100d3280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb91006cda0 tx=0x7fb90800b330 comp rx=0 tx=0).stop 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 -- 172.21.15.27:0/2074443787 shutdown_connections 2023-07-06T20:13:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b6100 0x7fb9100d3280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2074443787 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb9100d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2074443787 >> v2:172.21.15.153:3300/0 conn(0x7fb9100b5c90 0x7fb9100d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 -- 172.21.15.27:0/2074443787 >> 172.21.15.27:0/2074443787 conn(0x7fb91006c150 msgr2=0x7fb91006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 -- 172.21.15.27:0/2074443787 shutdown_connections 2023-07-06T20:13:54.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.138+0000 7fb9153ff700 1 -- 172.21.15.27:0/2074443787 wait complete. 2023-07-06T20:13:54.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.140+0000 7fb9153ff700 1 Processor -- start 2023-07-06T20:13:54.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.140+0000 7fb9153ff700 1 -- start start 2023-07-06T20:13:54.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb9153ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 0x7fb9100c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb9153ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9100b6100 0x7fb91015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb9153ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb910158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb9153ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9100c90f0 con 0x7fb9100b5c90 2023-07-06T20:13:54.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb9153ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb910149240 con 0x7fb9100b6100 2023-07-06T20:13:54.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb9153ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb910149520 con 0x7fb91015fca0 2023-07-06T20:13:54.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 0x7fb9100c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 0x7fb9100c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46514/0 (socket says 172.21.15.27:46514) 2023-07-06T20:13:54.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 -- 172.21.15.27:0/2686871376 learned_addr learned my addr 172.21.15.27:0/2686871376 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90f7fe700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb910158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90e7fc700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9100b6100 0x7fb91015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 -- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 msgr2=0x7fb910158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb910158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 -- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9100b6100 msgr2=0x7fb91015ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.141+0000 7fb90effd700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9100b6100 0x7fb91015ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb90effd700 1 -- 172.21.15.27:0/2686871376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb908009b30 con 0x7fb9100b5c90 2023-07-06T20:13:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb90f7fe700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb910158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb90effd700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 0x7fb9100c8bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb90000e810 tx=0x7fb90000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900004230 con 0x7fb9100b5c90 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb900004390 con 0x7fb9100b5c90 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb910149800 con 0x7fb9100b5c90 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9000036a0 con 0x7fb9100b5c90 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.142+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb910149dc0 con 0x7fb9100b5c90 2023-07-06T20:13:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.143+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9100008d0 con 0x7fb9100b5c90 2023-07-06T20:13:54.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.144+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb900003800 con 0x7fb9100b5c90 2023-07-06T20:13:54.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.146+0000 7fb8f77fe700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb8fc06ace0 0x7fb8fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.146+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb900088c80 con 0x7fb9100b5c90 2023-07-06T20:13:54.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.146+0000 7fb90e7fc700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb8fc06ace0 0x7fb8fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.147+0000 7fb90e7fc700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb8fc06ace0 0x7fb8fc06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb904009a50 tx=0x7fb904008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:54.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.150+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb900054070 con 0x7fb9100b5c90 2023-07-06T20:13:54.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.356+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb9100b7080 con 0x7fb9100b5c90 2023-07-06T20:13:54.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.356+0000 7fb8f77fe700 1 -- 172.21.15.27:0/2686871376 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb9000094c0 con 0x7fb9100b5c90 2023-07-06T20:13:54.358 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:13:54.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb8fc06ace0 msgr2=0x7fb8fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb8fc06ace0 0x7fb8fc06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb904009a50 tx=0x7fb904008040 comp rx=0 tx=0).stop 2023-07-06T20:13:54.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 msgr2=0x7fb9100c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 0x7fb9100c8bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb90000e810 tx=0x7fb90000ebd0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 shutdown_connections 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb8fc06ace0 0x7fb8fc06d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb91015fca0 0x7fb910158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9100b5c90 0x7fb9100c8bb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 --2- 172.21.15.27:0/2686871376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9100b6100 0x7fb91015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 >> 172.21.15.27:0/2686871376 conn(0x7fb91006c150 msgr2=0x7fb9100d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 shutdown_connections 2023-07-06T20:13:54.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.359+0000 7fb9153ff700 1 -- 172.21.15.27:0/2686871376 wait complete. 2023-07-06T20:13:54.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:13:54.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.579+0000 7fb8acdff700 1 Processor -- start 2023-07-06T20:13:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.580+0000 7fb8acdff700 1 -- start start 2023-07-06T20:13:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb8acdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb8a80b6110 0x7fb8a80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb8acdff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80cf780 con 0x7fb8a80b5c90 2023-07-06T20:13:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb8acdff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80cd490 con 0x7fb8a815fca0 2023-07-06T20:13:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb8acdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a80cd770 con 0x7fb8a80b6110 2023-07-06T20:13:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb89ffff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb8a80b5c90 0x7fb8a80d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46048/0 (socket says 172.21.15.27:46048) 2023-07-06T20:13:54.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb89ffff700 1 -- 172.21.15.27:0/4127708626 learned_addr learned my addr 172.21.15.27:0/4127708626 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.581+0000 7fb8a6d9d700 1 --2- 172.21.15.27:0/4127708626 >> v2:172.21.15.27:3300/0 conn(0x7fb8a80b6110 0x7fb8a80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a5d9b700 1 -- 172.21.15.27:0/4127708626 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377128563 0 0) 0x7fb8a80cf780 con 0x7fb8a80b5c90 2023-07-06T20:13:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a5d9b700 1 -- 172.21.15.27:0/4127708626 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8940035b0 con 0x7fb8a80b5c90 2023-07-06T20:13:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/4127708626 >> v1:172.21.15.153:6789/0 conn(0x7fb8a815fca0 legacy=0x7fb8a80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/4127708626 >> v1:172.21.15.27:6789/0 conn(0x7fb8a80b5c90 legacy=0x7fb8a80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/4127708626 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80cda50 con 0x7fb8a80b6110 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a6d9d700 1 --2- 172.21.15.27:0/4127708626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a80b6110 0x7fb8a80cfc10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009d90 tx=0x7fb8a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aed023970cbceb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a5d9b700 1 -- 172.21.15.27:0/4127708626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000e070 con 0x7fb8a80b6110 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a5d9b700 1 -- 172.21.15.27:0/4127708626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a000bb20 con 0x7fb8a80b6110 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.582+0000 7fb8a5d9b700 1 -- 172.21.15.27:0/4127708626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0012430 con 0x7fb8a80b6110 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 -- 172.21.15.27:0/4127708626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a80b6110 msgr2=0x7fb8a80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 --2- 172.21.15.27:0/4127708626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a80b6110 0x7fb8a80cfc10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009d90 tx=0x7fb8a000b330 comp rx=0 tx=0).stop 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 -- 172.21.15.27:0/4127708626 shutdown_connections 2023-07-06T20:13:54.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 --2- 172.21.15.27:0/4127708626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a80b6110 0x7fb8a80cfc10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 -- 172.21.15.27:0/4127708626 >> 172.21.15.27:0/4127708626 conn(0x7fb8a806c150 msgr2=0x7fb8a806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:54.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 -- 172.21.15.27:0/4127708626 shutdown_connections 2023-07-06T20:13:54.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.583+0000 7fb8acdff700 1 -- 172.21.15.27:0/4127708626 wait complete. 2023-07-06T20:13:54.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.585+0000 7fb8acdff700 1 Processor -- start 2023-07-06T20:13:54.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.586+0000 7fb8acdff700 1 -- start start 2023-07-06T20:13:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8acdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8a80b5c90 0x7fb8a80ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8acdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a80b6110 0x7fb8a80c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8acdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 0x7fb8a815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8acdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cbea0 con 0x7fb8a815fca0 2023-07-06T20:13:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8acdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cb220 con 0x7fb8a80b6110 2023-07-06T20:13:54.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8acdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cb500 con 0x7fb8a80b5c90 2023-07-06T20:13:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8a6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 0x7fb8a815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8a6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 0x7fb8a815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46536/0 (socket says 172.21.15.27:46536) 2023-07-06T20:13:54.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/2167094341 learned_addr learned my addr 172.21.15.27:0/2167094341 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:54.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb89ffff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a80b6110 0x7fb8a80c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8a80b5c90 msgr2=0x7fb8a80ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.587+0000 7fb8a659c700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8a80b5c90 0x7fb8a80ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a6d9d700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8a80b5c90 0x7fb8a80ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a80b6110 msgr2=0x7fb8a80c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a6d9d700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a80b6110 0x7fb8a80c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a6d9d700 1 -- 172.21.15.27:0/2167094341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0009a70 con 0x7fb8a815fca0 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb89ffff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a80b6110 0x7fb8a80c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a659c700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8a80b5c90 0x7fb8a80ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:13:54.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8a6d9d700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 0x7fb8a815b9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb8a806d0f0 tx=0x7fb8a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0012430 con 0x7fb8a815fca0 2023-07-06T20:13:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0020070 con 0x7fb8a815fca0 2023-07-06T20:13:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a80cb7e0 con 0x7fb8a815fca0 2023-07-06T20:13:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0004280 con 0x7fb8a815fca0 2023-07-06T20:13:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.588+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a80cae60 con 0x7fb8a815fca0 2023-07-06T20:13:54.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.589+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a80008d0 con 0x7fb8a815fca0 2023-07-06T20:13:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.590+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8a00043e0 con 0x7fb8a815fca0 2023-07-06T20:13:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.591+0000 7fb89effd700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb88806ace0 0x7fb88806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.591+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb8a0022070 con 0x7fb8a815fca0 2023-07-06T20:13:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.592+0000 7fb8a659c700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb88806ace0 0x7fb88806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:54.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.592+0000 7fb8a659c700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb88806ace0 0x7fb88806d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb898000f60 tx=0x7fb898009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:54.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.595+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8a00562a0 con 0x7fb8a815fca0 2023-07-06T20:13:54.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.844+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8a815af00 con 0x7fb8a815fca0 2023-07-06T20:13:54.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.844+0000 7fb89effd700 1 -- 172.21.15.27:0/2167094341 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fb8a002c090 con 0x7fb8a815fca0 2023-07-06T20:13:54.846 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:13:54.846 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4299,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb88806ace0 msgr2=0x7fb88806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb88806ace0 0x7fb88806d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb898000f60 tx=0x7fb898009040 comp rx=0 tx=0).stop 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 msgr2=0x7fb8a815b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 0x7fb8a815b9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb8a806d0f0 tx=0x7fb8a0004ad0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 shutdown_connections 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb88806ace0 0x7fb88806d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a815fca0 0x7fb8a815b9d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8a80b5c90 0x7fb8a80ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 --2- 172.21.15.27:0/2167094341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a80b6110 0x7fb8a80c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:54.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 >> 172.21.15.27:0/2167094341 conn(0x7fb8a806c150 msgr2=0x7fb8a815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:54.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 shutdown_connections 2023-07-06T20:13:54.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:54.847+0000 7fb8acdff700 1 -- 172.21.15.27:0/2167094341 wait complete. 2023-07-06T20:13:54.850 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2023-07-06T20:13:54.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:13:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.068+0000 7f613cbff700 1 Processor -- start 2023-07-06T20:13:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.068+0000 7f613cbff700 1 -- start start 2023-07-06T20:13:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.069+0000 7f613cbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f613815fca0 0x7f61380d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.069+0000 7f613cbff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 0x7f61380cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.069+0000 7f613cbff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61380cf780 con 0x7f61380b5c90 2023-07-06T20:13:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.069+0000 7f613cbff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61380cd490 con 0x7f61380b6100 2023-07-06T20:13:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.069+0000 7f613cbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61380cd770 con 0x7f613815fca0 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f613659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f613815fca0 0x7f61380d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f6136d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 0x7f61380cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f6136d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 0x7f61380cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54710/0 (socket says 172.21.15.27:54710) 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f6136d9d700 1 -- 172.21.15.27:0/2294468641 learned_addr learned my addr 172.21.15.27:0/2294468641 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f613659c700 1 -- 172.21.15.27:0/2294468641 >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 msgr2=0x7f61380cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f613659c700 1 --2- 172.21.15.27:0/2294468641 >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 0x7f61380cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f613659c700 1 -- 172.21.15.27:0/2294468641 >> v1:172.21.15.153:6789/0 conn(0x7f61380b5c90 legacy=0x7f61380d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f613659c700 1 -- 172.21.15.27:0/2294468641 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61380cda50 con 0x7f613815fca0 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f6136d9d700 1 --2- 172.21.15.27:0/2294468641 >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 0x7f61380cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.070+0000 7f6135d9b700 1 -- 172.21.15.27:0/2294468641 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638069813 0 0) 0x7f61380cf780 con 0x7f61380b5c90 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.071+0000 7f613659c700 1 --2- 172.21.15.27:0/2294468641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f613815fca0 0x7f61380d1420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6128009840 tx=0x7f6128009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86301dcae929d2de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.071+0000 7f6135d9b700 1 -- 172.21.15.27:0/2294468641 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128011830 con 0x7f613815fca0 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.071+0000 7f6135d9b700 1 -- 172.21.15.27:0/2294468641 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128011990 con 0x7f613815fca0 2023-07-06T20:13:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.071+0000 7f6135d9b700 1 -- 172.21.15.27:0/2294468641 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128011d20 con 0x7f613815fca0 2023-07-06T20:13:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 -- 172.21.15.27:0/2294468641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f613815fca0 msgr2=0x7f61380d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 --2- 172.21.15.27:0/2294468641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f613815fca0 0x7f61380d1420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6128009840 tx=0x7f6128009b50 comp rx=0 tx=0).stop 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 -- 172.21.15.27:0/2294468641 shutdown_connections 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 --2- 172.21.15.27:0/2294468641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f613815fca0 0x7f61380d1420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 --2- 172.21.15.27:0/2294468641 >> v2:172.21.15.153:3300/0 conn(0x7f61380b6100 0x7f61380cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 -- 172.21.15.27:0/2294468641 >> 172.21.15.27:0/2294468641 conn(0x7f613806c150 msgr2=0x7f613806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 -- 172.21.15.27:0/2294468641 shutdown_connections 2023-07-06T20:13:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.072+0000 7f613cbff700 1 -- 172.21.15.27:0/2294468641 wait complete. 2023-07-06T20:13:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.073+0000 7f613cbff700 1 Processor -- start 2023-07-06T20:13:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.074+0000 7f613cbff700 1 -- start start 2023-07-06T20:13:55.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613cbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61380b5c90 0x7f61380c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613cbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 0x7f61380b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613cbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f613815fca0 0x7f61380d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613cbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380b4ae0 con 0x7f61380b6100 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613cbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380b2bf0 con 0x7f613815fca0 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613cbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61380b2ed0 con 0x7f61380b5c90 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 0x7f61380b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 0x7f61380b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46550/0 (socket says 172.21.15.27:46550) 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 -- 172.21.15.27:0/1989319989 learned_addr learned my addr 172.21.15.27:0/1989319989 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f613659c700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61380b5c90 0x7f61380c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f6136d9d700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f613815fca0 0x7f61380d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 -- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61380b5c90 msgr2=0x7f61380c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61380b5c90 0x7f61380c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 -- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f613815fca0 msgr2=0x7f61380d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f613815fca0 0x7f61380d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.075+0000 7f612ffff700 1 -- 172.21.15.27:0/1989319989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6130009b30 con 0x7f61380b6100 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.076+0000 7f612ffff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 0x7f61380b50a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61200011e0 tx=0x7f612000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.076+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61200054e0 con 0x7f61380b6100 2023-07-06T20:13:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.076+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6120003680 con 0x7f61380b6100 2023-07-06T20:13:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.076+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120003a30 con 0x7f61380b6100 2023-07-06T20:13:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.076+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61280094a0 con 0x7f61380b6100 2023-07-06T20:13:55.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.076+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61380b3490 con 0x7f61380b6100 2023-07-06T20:13:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.077+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61380008d0 con 0x7f61380b6100 2023-07-06T20:13:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.078+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6120003b90 con 0x7f61380b6100 2023-07-06T20:13:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.079+0000 7f612effd700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f611806ace0 0x7f611806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.079+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6120088590 con 0x7f61380b6100 2023-07-06T20:13:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.080+0000 7f613659c700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f611806ace0 0x7f611806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.080+0000 7f613659c700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f611806ace0 0x7f611806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61280090b0 tx=0x7f6128008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.083+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6120053980 con 0x7f61380b6100 2023-07-06T20:13:55.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.293+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61380d1200 con 0x7f61380b6100 2023-07-06T20:13:55.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.294+0000 7f612effd700 1 -- 172.21.15.27:0/1989319989 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14612 (secure 0 0 0) 0x7f6120056fa0 con 0x7f61380b6100 2023-07-06T20:13:55.296 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:13:55.296 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:53.968195+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:13:55.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f611806ace0 msgr2=0x7f611806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f611806ace0 0x7f611806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61280090b0 tx=0x7f6128008030 comp rx=0 tx=0).stop 2023-07-06T20:13:55.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 msgr2=0x7f61380b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 0x7f61380b50a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61200011e0 tx=0x7f612000ca30 comp rx=0 tx=0).stop 2023-07-06T20:13:55.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 shutdown_connections 2023-07-06T20:13:55.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f611806ace0 0x7f611806d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f613815fca0 0x7f61380d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61380b5c90 0x7f61380c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 --2- 172.21.15.27:0/1989319989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61380b6100 0x7f61380b50a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 >> 172.21.15.27:0/1989319989 conn(0x7f613806c150 msgr2=0x7f613806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:55.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 shutdown_connections 2023-07-06T20:13:55.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.296+0000 7f613cbff700 1 -- 172.21.15.27:0/1989319989 wait complete. 2023-07-06T20:13:55.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:13:55.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:13:55.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.519+0000 7f88115ff700 1 Processor -- start 2023-07-06T20:13:55.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.520+0000 7f88115ff700 1 -- start start 2023-07-06T20:13:55.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f88115ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 0x7f880c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f88115ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f880c0cf0a0 0x7f880c15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f88115ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880c0cf540 con 0x7f880c15a390 2023-07-06T20:13:55.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f88115ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880c0cf820 con 0x7f880c0cf0a0 2023-07-06T20:13:55.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f88115ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0cdcb0 con 0x7f880c159fc0 2023-07-06T20:13:55.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f8803fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f880c0cf0a0 0x7f880c15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.521+0000 7f880affd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 0x7f880c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 0x7f880c0d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:50446/0 (socket says 172.21.15.27:50446) 2023-07-06T20:13:55.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 -- 172.21.15.27:0/2665913354 learned_addr learned my addr 172.21.15.27:0/2665913354 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 -- 172.21.15.27:0/2665913354 >> v2:172.21.15.153:3300/0 conn(0x7f880c0cf0a0 msgr2=0x7f880c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 --2- 172.21.15.27:0/2665913354 >> v2:172.21.15.153:3300/0 conn(0x7f880c0cf0a0 0x7f880c15d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 -- 172.21.15.27:0/2665913354 >> v1:172.21.15.153:6789/0 conn(0x7f880c15a390 legacy=0x7f880c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 -- 172.21.15.27:0/2665913354 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0cd660 con 0x7f880c159fc0 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880a7fc700 1 -- 172.21.15.27:0/2665913354 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782299358 0 0) 0x7f880c0cf540 con 0x7f880c15a390 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880affd700 1 --2- 172.21.15.27:0/2665913354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 0x7f880c0d29b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0098f0 tx=0x7f87fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff36c95acee35ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880a7fc700 1 -- 172.21.15.27:0/2665913354 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc011830 con 0x7f880c159fc0 2023-07-06T20:13:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.522+0000 7f880a7fc700 1 -- 172.21.15.27:0/2665913354 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc011990 con 0x7f880c159fc0 2023-07-06T20:13:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f880a7fc700 1 -- 172.21.15.27:0/2665913354 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc011d20 con 0x7f880c159fc0 2023-07-06T20:13:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 -- 172.21.15.27:0/2665913354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 msgr2=0x7f880c0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 --2- 172.21.15.27:0/2665913354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 0x7f880c0d29b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0098f0 tx=0x7f87fc009c00 comp rx=0 tx=0).stop 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 -- 172.21.15.27:0/2665913354 shutdown_connections 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 --2- 172.21.15.27:0/2665913354 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c159fc0 0x7f880c0d29b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 --2- 172.21.15.27:0/2665913354 >> v2:172.21.15.153:3300/0 conn(0x7f880c0cf0a0 0x7f880c15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 -- 172.21.15.27:0/2665913354 >> 172.21.15.27:0/2665913354 conn(0x7f880c06c1e0 msgr2=0x7f880c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 -- 172.21.15.27:0/2665913354 shutdown_connections 2023-07-06T20:13:55.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.523+0000 7f88115ff700 1 -- 172.21.15.27:0/2665913354 wait complete. 2023-07-06T20:13:55.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.525+0000 7f88115ff700 1 Processor -- start 2023-07-06T20:13:55.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.525+0000 7f88115ff700 1 -- start start 2023-07-06T20:13:55.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f88115ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f880c0cf0a0 0x7f880c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f88115ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 0x7f880c0cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f88115ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c15a390 0x7f880c0b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f88115ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0b8cb0 con 0x7f880c159fc0 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f88115ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c1e4840 con 0x7f880c0cf0a0 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f88115ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c1e3f10 con 0x7f880c15a390 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f8803fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 0x7f880c0cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f8803fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 0x7f880c0cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46558/0 (socket says 172.21.15.27:46558) 2023-07-06T20:13:55.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f8803fff700 1 -- 172.21.15.27:0/557443260 learned_addr learned my addr 172.21.15.27:0/557443260 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f880affd700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f880c0cf0a0 0x7f880c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.526+0000 7f880b7fe700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c15a390 0x7f880c0b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f8803fff700 1 -- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c15a390 msgr2=0x7f880c0b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f8803fff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c15a390 0x7f880c0b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f8803fff700 1 -- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f880c0cf0a0 msgr2=0x7f880c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f8803fff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f880c0cf0a0 0x7f880c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f8803fff700 1 -- 172.21.15.27:0/557443260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc009550 con 0x7f880c159fc0 2023-07-06T20:13:55.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f8803fff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 0x7f880c0cb200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87f400e980 tx=0x7f87f400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4009710 con 0x7f880c159fc0 2023-07-06T20:13:55.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f400c690 con 0x7f880c159fc0 2023-07-06T20:13:55.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400ca40 con 0x7f880c159fc0 2023-07-06T20:13:55.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.527+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c1e41f0 con 0x7f880c159fc0 2023-07-06T20:13:55.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.528+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c152690 con 0x7f880c159fc0 2023-07-06T20:13:55.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.529+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c1e49f0 con 0x7f880c159fc0 2023-07-06T20:13:55.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.530+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87f400cba0 con 0x7f880c159fc0 2023-07-06T20:13:55.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.531+0000 7f88037fe700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f87ec06ace0 0x7f87ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.531+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f87f4089130 con 0x7f880c159fc0 2023-07-06T20:13:55.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.531+0000 7f880affd700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f87ec06ace0 0x7f87ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.532+0000 7f880affd700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f87ec06ace0 0x7f87ec06d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009c40 tx=0x7f87fc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.535+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f880c1e49f0 con 0x7f880c159fc0 2023-07-06T20:13:55.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.740+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f880c068190 con 0x7f880c159fc0 2023-07-06T20:13:55.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.741+0000 7f88037fe700 1 -- 172.21.15.27:0/557443260 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87f4057f20 con 0x7f880c159fc0 2023-07-06T20:13:55.742 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:13:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f87ec06ace0 msgr2=0x7f87ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f87ec06ace0 0x7f87ec06d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009c40 tx=0x7f87fc008030 comp rx=0 tx=0).stop 2023-07-06T20:13:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 msgr2=0x7f880c0cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 0x7f880c0cb200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87f400e980 tx=0x7f87f400ec90 comp rx=0 tx=0).stop 2023-07-06T20:13:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 shutdown_connections 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f87ec06ace0 0x7f87ec06d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f880c15a390 0x7f880c0b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f880c0cf0a0 0x7f880c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 --2- 172.21.15.27:0/557443260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c159fc0 0x7f880c0cb200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 >> 172.21.15.27:0/557443260 conn(0x7f880c06c1e0 msgr2=0x7f880c1e7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 shutdown_connections 2023-07-06T20:13:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.743+0000 7f88115ff700 1 -- 172.21.15.27:0/557443260 wait complete. 2023-07-06T20:13:55.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:13:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.968+0000 7f94d13ff700 1 Processor -- start 2023-07-06T20:13:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.969+0000 7f94d13ff700 1 -- start start 2023-07-06T20:13:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94d13ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f94cc15fca0 0x7f94cc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94d13ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94cc0b6110 0x7f94cc0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94d13ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94cc0cf780 con 0x7f94cc0b5c90 2023-07-06T20:13:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94d13ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc0cd490 con 0x7f94cc0b6110 2023-07-06T20:13:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94d13ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc0cd770 con 0x7f94cc15fca0 2023-07-06T20:13:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94cb7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94cc0b6110 0x7f94cc0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94ca7fc700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f94cc0b5c90 0x7f94cc0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46054/0 (socket says 172.21.15.27:46054) 2023-07-06T20:13:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94ca7fc700 1 -- 172.21.15.27:0/2594838532 learned_addr learned my addr 172.21.15.27:0/2594838532 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.970+0000 7f94caffd700 1 --2- 172.21.15.27:0/2594838532 >> v2:172.21.15.153:3300/0 conn(0x7f94cc15fca0 0x7f94cc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94cb7fe700 1 -- 172.21.15.27:0/2594838532 >> v2:172.21.15.153:3300/0 conn(0x7f94cc15fca0 msgr2=0x7f94cc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94cb7fe700 1 --2- 172.21.15.27:0/2594838532 >> v2:172.21.15.153:3300/0 conn(0x7f94cc15fca0 0x7f94cc0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94cb7fe700 1 -- 172.21.15.27:0/2594838532 >> v1:172.21.15.27:6789/0 conn(0x7f94cc0b5c90 legacy=0x7f94cc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94cb7fe700 1 -- 172.21.15.27:0/2594838532 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0cda50 con 0x7f94cc0b6110 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94c9ffb700 1 -- 172.21.15.27:0/2594838532 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026062391 0 0) 0x7f94cc0cf780 con 0x7f94cc0b5c90 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94cb7fe700 1 --2- 172.21.15.27:0/2594838532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b6110 0x7f94cc0cfc10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0d0930 tx=0x7f94c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a16ee3a869c43b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94c9ffb700 1 -- 172.21.15.27:0/2594838532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c400e070 con 0x7f94cc0b6110 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94c9ffb700 1 -- 172.21.15.27:0/2594838532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c400bbd0 con 0x7f94cc0b6110 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.971+0000 7f94c9ffb700 1 -- 172.21.15.27:0/2594838532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4012430 con 0x7f94cc0b6110 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 -- 172.21.15.27:0/2594838532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b6110 msgr2=0x7f94cc0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 --2- 172.21.15.27:0/2594838532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b6110 0x7f94cc0cfc10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0d0930 tx=0x7f94c400b330 comp rx=0 tx=0).stop 2023-07-06T20:13:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 -- 172.21.15.27:0/2594838532 shutdown_connections 2023-07-06T20:13:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 --2- 172.21.15.27:0/2594838532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b6110 0x7f94cc0cfc10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 --2- 172.21.15.27:0/2594838532 >> v2:172.21.15.153:3300/0 conn(0x7f94cc15fca0 0x7f94cc0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 -- 172.21.15.27:0/2594838532 >> 172.21.15.27:0/2594838532 conn(0x7f94cc06c150 msgr2=0x7f94cc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 -- 172.21.15.27:0/2594838532 shutdown_connections 2023-07-06T20:13:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.972+0000 7f94d13ff700 1 -- 172.21.15.27:0/2594838532 wait complete. 2023-07-06T20:13:55.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.974+0000 7f94d13ff700 1 Processor -- start 2023-07-06T20:13:55.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.974+0000 7f94d13ff700 1 -- start start 2023-07-06T20:13:55.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94d13ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 0x7f94cc157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94d13ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b6110 0x7f94cc0b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94d13ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc15fca0 0x7f94cc0b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94d13ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc158e70 con 0x7f94cc0b5c90 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94d13ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc158790 con 0x7f94cc15fca0 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94d13ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc158a70 con 0x7f94cc0b6110 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 0x7f94cc157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 0x7f94cc157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46574/0 (socket says 172.21.15.27:46574) 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 -- 172.21.15.27:0/3756230382 learned_addr learned my addr 172.21.15.27:0/3756230382 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94ca7fc700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b6110 0x7f94cc0b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94cb7fe700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc15fca0 0x7f94cc0b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 -- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b6110 msgr2=0x7f94cc0b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b6110 0x7f94cc0b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 -- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc15fca0 msgr2=0x7f94cc0b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc15fca0 0x7f94cc0b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94caffd700 1 -- 172.21.15.27:0/3756230382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4009a70 con 0x7f94cc0b5c90 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94ca7fc700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b6110 0x7f94cc0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:13:55.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.975+0000 7f94cb7fe700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc15fca0 0x7f94cc0b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:13:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.976+0000 7f94caffd700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 0x7f94cc157cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d800 tx=0x7f94bc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.976+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0125f0 con 0x7f94cc0b5c90 2023-07-06T20:13:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.976+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc012750 con 0x7f94cc0b5c90 2023-07-06T20:13:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.976+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc1590c0 con 0x7f94cc0b5c90 2023-07-06T20:13:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.976+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc012b00 con 0x7f94cc0b5c90 2023-07-06T20:13:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.976+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc0c6bc0 con 0x7f94cc0b5c90 2023-07-06T20:13:55.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.977+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc0008d0 con 0x7f94cc0b5c90 2023-07-06T20:13:55.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.978+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94bc012c60 con 0x7f94cc0b5c90 2023-07-06T20:13:55.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.980+0000 7f94b37fe700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94b806ace0 0x7f94b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.980+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f94bc089520 con 0x7f94cc0b5c90 2023-07-06T20:13:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.980+0000 7f94ca7fc700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94b806ace0 0x7f94b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.981+0000 7f94ca7fc700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94b806ace0 0x7f94b806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94c0000f60 tx=0x7f94c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:55.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:55.984+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94bc054910 con 0x7f94cc0b5c90 2023-07-06T20:13:56.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.188+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f94cc0cec70 con 0x7f94cc0b5c90 2023-07-06T20:13:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.188+0000 7f94b37fe700 1 -- 172.21.15.27:0/3756230382 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f94bc057f30 con 0x7f94cc0b5c90 2023-07-06T20:13:56.190 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:13:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.190+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94b806ace0 msgr2=0x7f94b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.190+0000 7f94d13ff700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94b806ace0 0x7f94b806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94c0000f60 tx=0x7f94c0009040 comp rx=0 tx=0).stop 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.190+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 msgr2=0x7f94cc157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.190+0000 7f94d13ff700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 0x7f94cc157cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d800 tx=0x7f94bc00dbc0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 shutdown_connections 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94b806ace0 0x7f94b806d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc15fca0 0x7f94cc0b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94cc0b5c90 0x7f94cc157cd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 --2- 172.21.15.27:0/3756230382 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b6110 0x7f94cc0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 >> 172.21.15.27:0/3756230382 conn(0x7f94cc06c150 msgr2=0x7f94cc0d2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 shutdown_connections 2023-07-06T20:13:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.191+0000 7f94d13ff700 1 -- 172.21.15.27:0/3756230382 wait complete. 2023-07-06T20:13:56.210 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T20:13:56.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.410+0000 7f9c4fdff700 1 Processor -- start 2023-07-06T20:13:56.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.410+0000 7f9c4fdff700 1 -- start start 2023-07-06T20:13:56.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4fdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9c480b5c90 0x7f9c480d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4fdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b6110 0x7f9c480d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4fdff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c480cf780 con 0x7f9c4815fca0 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4fdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c480cd490 con 0x7f9c480b5c90 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4fdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480cd770 con 0x7f9c480b6110 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9c480b5c90 0x7f9c480d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9c480b5c90 0x7f9c480d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46586/0 (socket says 172.21.15.27:46586) 2023-07-06T20:13:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 -- 172.21.15.27:0/608189464 learned_addr learned my addr 172.21.15.27:0/608189464 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4e39c700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b6110 0x7f9c480d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 -- 172.21.15.27:0/608189464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b6110 msgr2=0x7f9c480d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b6110 0x7f9c480d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 -- 172.21.15.27:0/608189464 >> v1:172.21.15.153:6789/0 conn(0x7f9c4815fca0 legacy=0x7f9c480d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4d39a700 1 -- 172.21.15.27:0/608189464 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480cda50 con 0x7f9c480b5c90 2023-07-06T20:13:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.412+0000 7f9c4e39c700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b6110 0x7f9c480d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4d39a700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c480b5c90 0x7f9c480d2d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c380098f0 tx=0x7f9c38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636084531d9e672c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4cb99700 1 -- 172.21.15.27:0/608189464 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800b8a0 con 0x7f9c480b5c90 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4cb99700 1 -- 172.21.15.27:0/608189464 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800ba00 con 0x7f9c480b5c90 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4cb99700 1 -- 172.21.15.27:0/608189464 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800bd90 con 0x7f9c480b5c90 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4fdff700 1 -- 172.21.15.27:0/608189464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c480b5c90 msgr2=0x7f9c480d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c480b5c90 0x7f9c480d2d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c380098f0 tx=0x7f9c38009c00 comp rx=0 tx=0).stop 2023-07-06T20:13:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4fdff700 1 -- 172.21.15.27:0/608189464 shutdown_connections 2023-07-06T20:13:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b6110 0x7f9c480d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/608189464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c480b5c90 0x7f9c480d2d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.413+0000 7f9c4fdff700 1 -- 172.21.15.27:0/608189464 >> 172.21.15.27:0/608189464 conn(0x7f9c4806c150 msgr2=0x7f9c4806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.414+0000 7f9c4fdff700 1 -- 172.21.15.27:0/608189464 shutdown_connections 2023-07-06T20:13:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.414+0000 7f9c4fdff700 1 -- 172.21.15.27:0/608189464 wait complete. 2023-07-06T20:13:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.416+0000 7f9c4fdff700 1 Processor -- start 2023-07-06T20:13:56.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.416+0000 7f9c4fdff700 1 -- start start 2023-07-06T20:13:56.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.417+0000 7f9c4fdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b5c90 0x7f9c48157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.417+0000 7f9c4fdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c480b6110 0x7f9c480c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.417+0000 7f9c4fdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 0x7f9c481591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.417+0000 7f9c4fdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480cbdb0 con 0x7f9c4815fca0 2023-07-06T20:13:56.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.417+0000 7f9c4fdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48157a60 con 0x7f9c480b6110 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4fdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480cbfc0 con 0x7f9c480b5c90 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 0x7f9c481591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 0x7f9c481591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46590/0 (socket says 172.21.15.27:46590) 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 -- 172.21.15.27:0/2460456287 learned_addr learned my addr 172.21.15.27:0/2460456287 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4db9b700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b5c90 0x7f9c48157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4d39a700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c480b6110 0x7f9c480c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 -- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b5c90 msgr2=0x7f9c48157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b5c90 0x7f9c48157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 -- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c480b6110 msgr2=0x7f9c480c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c480b6110 0x7f9c480c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 -- 172.21.15.27:0/2460456287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38009550 con 0x7f9c4815fca0 2023-07-06T20:13:56.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.418+0000 7f9c4e39c700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 0x7f9c481591f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c40007ef0 tx=0x7f9c4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.419+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000dd80 con 0x7f9c4815fca0 2023-07-06T20:13:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.419+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4000f040 con 0x7f9c4815fca0 2023-07-06T20:13:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.419+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40003c60 con 0x7f9c4815fca0 2023-07-06T20:13:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.419+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c480cc2a0 con 0x7f9c4815fca0 2023-07-06T20:13:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.419+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c480cc860 con 0x7f9c4815fca0 2023-07-06T20:13:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.420+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c480008d0 con 0x7f9c4815fca0 2023-07-06T20:13:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.421+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c40004030 con 0x7f9c4815fca0 2023-07-06T20:13:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.422+0000 7f9c3e7fc700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9c2c06ace0 0x7f9c2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.422+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9c400891f0 con 0x7f9c4815fca0 2023-07-06T20:13:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.422+0000 7f9c4db9b700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9c2c06ace0 0x7f9c2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.423+0000 7f9c4db9b700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9c2c06ace0 0x7f9c2c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c44007fd0 tx=0x7f9c44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.426+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9c400545e0 con 0x7f9c4815fca0 2023-07-06T20:13:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.636+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9c4815a9b0 con 0x7f9c4815fca0 2023-07-06T20:13:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.636+0000 7f9c3e7fc700 1 -- 172.21.15.27:0/2460456287 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9c40057c00 con 0x7f9c4815fca0 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:13:56.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:13:56.640 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 2, 2023-07-06T20:13:56.640 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": "rbd" 2023-07-06T20:13:56.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:13:56.640 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-07-06T20:13:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9c2c06ace0 msgr2=0x7f9c2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9c2c06ace0 0x7f9c2c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c44007fd0 tx=0x7f9c44009040 comp rx=0 tx=0).stop 2023-07-06T20:13:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 msgr2=0x7f9c481591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 0x7f9c481591f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c40007ef0 tx=0x7f9c4000d3b0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 shutdown_connections 2023-07-06T20:13:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9c2c06ace0 0x7f9c2c06d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4815fca0 0x7f9c481591f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9c480b5c90 0x7f9c48157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 --2- 172.21.15.27:0/2460456287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c480b6110 0x7f9c480c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 >> 172.21.15.27:0/2460456287 conn(0x7f9c4806c150 msgr2=0x7f9c480b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 shutdown_connections 2023-07-06T20:13:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.639+0000 7f9c4fdff700 1 -- 172.21.15.27:0/2460456287 wait complete. 2023-07-06T20:13:56.658 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T20:13:56.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T20:13:56.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.864+0000 7f766bbff700 1 Processor -- start 2023-07-06T20:13:56.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.865+0000 7f766bbff700 1 -- start start 2023-07-06T20:13:56.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766bbff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f76640cfe40 0x7f76640cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766bbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f76640d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766bbff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7664160cc0 con 0x7f76640b8440 2023-07-06T20:13:56.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766bbff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7664160ae0 con 0x7f76640cfe40 2023-07-06T20:13:56.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766bbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7664160f80 con 0x7f76640b2180 2023-07-06T20:13:56.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766919a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f76640cfe40 0x7f76640cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766a19c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f76640d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766999b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f76640b8440 0x7f76640cf730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:34286/0 (socket says 172.21.15.27:34286) 2023-07-06T20:13:56.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.866+0000 7f766999b700 1 -- 172.21.15.27:0/748857097 learned_addr learned my addr 172.21.15.27:0/748857097 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:13:56.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f7668999700 1 -- 172.21.15.27:0/748857097 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133053762 0 0) 0x7f7664160cc0 con 0x7f76640b8440 2023-07-06T20:13:56.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f7668999700 1 -- 172.21.15.27:0/748857097 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76500035b0 con 0x7f76640b8440 2023-07-06T20:13:56.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f766a19c700 1 -- 172.21.15.27:0/748857097 >> v2:172.21.15.153:3300/0 conn(0x7f76640cfe40 msgr2=0x7f76640cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f766a19c700 1 --2- 172.21.15.27:0/748857097 >> v2:172.21.15.153:3300/0 conn(0x7f76640cfe40 0x7f76640cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f766a19c700 1 -- 172.21.15.27:0/748857097 >> v1:172.21.15.153:6789/0 conn(0x7f76640b8440 legacy=0x7f76640cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f766a19c700 1 -- 172.21.15.27:0/748857097 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640cf0e0 con 0x7f76640b2180 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f766919a700 1 --2- 172.21.15.27:0/748857097 >> v2:172.21.15.153:3300/0 conn(0x7f76640cfe40 0x7f76640cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f766a19c700 1 --2- 172.21.15.27:0/748857097 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f76640d0960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f765c009d90 tx=0x7f765c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bae5273b496962c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f7668999700 1 -- 172.21.15.27:0/748857097 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00e070 con 0x7f76640b2180 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f7668999700 1 -- 172.21.15.27:0/748857097 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c00bb20 con 0x7f76640b2180 2023-07-06T20:13:56.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.867+0000 7f7668999700 1 -- 172.21.15.27:0/748857097 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c012430 con 0x7f76640b2180 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 -- 172.21.15.27:0/748857097 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 msgr2=0x7f76640d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 --2- 172.21.15.27:0/748857097 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f76640d0960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f765c009d90 tx=0x7f765c00b330 comp rx=0 tx=0).stop 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 -- 172.21.15.27:0/748857097 shutdown_connections 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 --2- 172.21.15.27:0/748857097 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f76640d0960 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 --2- 172.21.15.27:0/748857097 >> v2:172.21.15.153:3300/0 conn(0x7f76640cfe40 0x7f76640cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 -- 172.21.15.27:0/748857097 >> 172.21.15.27:0/748857097 conn(0x7f766406be10 msgr2=0x7f7664071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 -- 172.21.15.27:0/748857097 shutdown_connections 2023-07-06T20:13:56.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.868+0000 7f766bbff700 1 -- 172.21.15.27:0/748857097 wait complete. 2023-07-06T20:13:56.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.870+0000 7f766bbff700 1 Processor -- start 2023-07-06T20:13:56.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.870+0000 7f766bbff700 1 -- start start 2023-07-06T20:13:56.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766bbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f7664150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766bbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76640b8440 0x7f76640c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766bbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 0x7f76640c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766bbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640cc8c0 con 0x7f76640cfe40 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766bbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640cb3c0 con 0x7f76640b8440 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766bbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76640c6a20 con 0x7f76640b2180 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766a19c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 0x7f76640c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766a19c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 0x7f76640c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46598/0 (socket says 172.21.15.27:46598) 2023-07-06T20:13:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.871+0000 7f766a19c700 1 -- 172.21.15.27:0/1171477267 learned_addr learned my addr 172.21.15.27:0/1171477267 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766999b700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f7664150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766919a700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76640b8440 0x7f76640c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766a19c700 1 -- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 msgr2=0x7f7664150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766a19c700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f7664150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766a19c700 1 -- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76640b8440 msgr2=0x7f76640c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766a19c700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76640b8440 0x7f76640c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766a19c700 1 -- 172.21.15.27:0/1171477267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540094a0 con 0x7f76640cfe40 2023-07-06T20:13:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766a19c700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 0x7f76640c9820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f765c014040 tx=0x7f765c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0129f0 con 0x7f76640cfe40 2023-07-06T20:13:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c020070 con 0x7f76640cfe40 2023-07-06T20:13:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c012db0 con 0x7f76640cfe40 2023-07-06T20:13:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.872+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c009a70 con 0x7f76640cfe40 2023-07-06T20:13:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.873+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76640cae00 con 0x7f76640cfe40 2023-07-06T20:13:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.874+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76640c6bd0 con 0x7f76640cfe40 2023-07-06T20:13:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.874+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f765c004040 con 0x7f76640cfe40 2023-07-06T20:13:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.875+0000 7f765a7fc700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f764806ace0 0x7f764806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.876+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f765c022070 con 0x7f76640cfe40 2023-07-06T20:13:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.876+0000 7f766999b700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f764806ace0 0x7f764806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.876+0000 7f766999b700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f764806ace0 0x7f764806d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f76600011e0 tx=0x7f7660009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:56.880+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f76640c6bd0 con 0x7f76640cfe40 2023-07-06T20:13:57.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:57.091+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76640cda60 con 0x7f76640cfe40 2023-07-06T20:13:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:57.998+0000 7f765a7fc700 1 -- 172.21.15.27:0/1171477267 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f765c02c0b0 con 0x7f76640cfe40 2023-07-06T20:13:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f764806ace0 msgr2=0x7f764806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f764806ace0 0x7f764806d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f76600011e0 tx=0x7f7660009040 comp rx=0 tx=0).stop 2023-07-06T20:13:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 msgr2=0x7f76640c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 0x7f76640c9820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f765c014040 tx=0x7f765c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 shutdown_connections 2023-07-06T20:13:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f764806ace0 0x7f764806d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76640cfe40 0x7f76640c9820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76640b2180 0x7f7664150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 --2- 172.21.15.27:0/1171477267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76640b8440 0x7f76640c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.004+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 >> 172.21.15.27:0/1171477267 conn(0x7f766406be10 msgr2=0x7f76640b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.005+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 shutdown_connections 2023-07-06T20:13:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.005+0000 7f766bbff700 1 -- 172.21.15.27:0/1171477267 wait complete. 2023-07-06T20:13:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-07-06T20:13:58.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T20:13:58.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.242+0000 7f3d037ff700 1 Processor -- start 2023-07-06T20:13:58.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.242+0000 7f3d037ff700 1 -- start start 2023-07-06T20:13:58.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.243+0000 7f3d037ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3cfc0b5c90 0x7f3cfc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:58.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.243+0000 7f3d037ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 0x7f3cfc0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:58.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.243+0000 7f3d037ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0cf780 con 0x7f3cfc15fca0 2023-07-06T20:13:58.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.243+0000 7f3d037ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc0cd490 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.243+0000 7f3d037ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0cd770 con 0x7f3cfc0b6110 2023-07-06T20:13:58.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 0x7f3cfc0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d00d9a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3cfc0b5c90 0x7f3cfc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 0x7f3cfc0d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:50502/0 (socket says 172.21.15.27:50502) 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 -- 172.21.15.27:0/37763265 learned_addr learned my addr 172.21.15.27:0/37763265 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 -- 172.21.15.27:0/37763265 >> v2:172.21.15.153:3300/0 conn(0x7f3cfc0b5c90 msgr2=0x7f3cfc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 --2- 172.21.15.27:0/37763265 >> v2:172.21.15.153:3300/0 conn(0x7f3cfc0b5c90 0x7f3cfc0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 -- 172.21.15.27:0/37763265 >> v1:172.21.15.153:6789/0 conn(0x7f3cfc15fca0 legacy=0x7f3cfc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d01d9c700 1 -- 172.21.15.27:0/37763265 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0cda50 con 0x7f3cfc0b6110 2023-07-06T20:13:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3d00d9a700 1 --2- 172.21.15.27:0/37763265 >> v2:172.21.15.153:3300/0 conn(0x7f3cfc0b5c90 0x7f3cfc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.244+0000 7f3cf3fff700 1 -- 172.21.15.27:0/37763265 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562586184 0 0) 0x7f3cfc0cf780 con 0x7f3cfc15fca0 2023-07-06T20:13:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d01d9c700 1 --2- 172.21.15.27:0/37763265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 0x7f3cfc0d3280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40098f0 tx=0x7f3cf4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e83f477b859d3a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3cf3fff700 1 -- 172.21.15.27:0/37763265 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4011830 con 0x7f3cfc0b6110 2023-07-06T20:13:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3cf3fff700 1 -- 172.21.15.27:0/37763265 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf4011990 con 0x7f3cfc0b6110 2023-07-06T20:13:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3cf3fff700 1 -- 172.21.15.27:0/37763265 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4011d20 con 0x7f3cfc0b6110 2023-07-06T20:13:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d037ff700 1 -- 172.21.15.27:0/37763265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 msgr2=0x7f3cfc0d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d037ff700 1 --2- 172.21.15.27:0/37763265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 0x7f3cfc0d3280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40098f0 tx=0x7f3cf4009c00 comp rx=0 tx=0).stop 2023-07-06T20:13:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d037ff700 1 -- 172.21.15.27:0/37763265 shutdown_connections 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d037ff700 1 --2- 172.21.15.27:0/37763265 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc0b6110 0x7f3cfc0d3280 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d037ff700 1 --2- 172.21.15.27:0/37763265 >> v2:172.21.15.153:3300/0 conn(0x7f3cfc0b5c90 0x7f3cfc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.245+0000 7f3d037ff700 1 -- 172.21.15.27:0/37763265 >> 172.21.15.27:0/37763265 conn(0x7f3cfc06c150 msgr2=0x7f3cfc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.246+0000 7f3d037ff700 1 -- 172.21.15.27:0/37763265 shutdown_connections 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.246+0000 7f3d037ff700 1 -- 172.21.15.27:0/37763265 wait complete. 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.247+0000 7f3d037ff700 1 Processor -- start 2023-07-06T20:13:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.247+0000 7f3d037ff700 1 -- start start 2023-07-06T20:13:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.248+0000 7f3d037ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 0x7f3cfc152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.248+0000 7f3d037ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cfc0b6110 0x7f3cfc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.248+0000 7f3d037ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc15fca0 0x7f3cfc1545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.248+0000 7f3d037ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc156a30 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.248+0000 7f3d037ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b3d00 con 0x7f3cfc0b6110 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.248+0000 7f3d037ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b30f0 con 0x7f3cfc15fca0 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 0x7f3cfc152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 0x7f3cfc152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46604/0 (socket says 172.21.15.27:46604) 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 -- 172.21.15.27:0/1234391910 learned_addr learned my addr 172.21.15.27:0/1234391910 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d01d9c700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc15fca0 0x7f3cfc1545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d00d9a700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cfc0b6110 0x7f3cfc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 -- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc15fca0 msgr2=0x7f3cfc1545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc15fca0 0x7f3cfc1545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 -- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cfc0b6110 msgr2=0x7f3cfc0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cfc0b6110 0x7f3cfc0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 -- 172.21.15.27:0/1234391910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0094a0 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.249+0000 7f3d0159b700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 0x7f3cfc152e90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c8c0 tx=0x7f3cf800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.250+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8003e30 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.250+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf8005070 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.250+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8005420 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.250+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf4009550 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.250+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc0b36b0 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.251+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc0008d0 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.252+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3cf8005580 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.253+0000 7f3cf1ffb700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ce006ace0 0x7f3ce006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.253+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3cf8088900 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.253+0000 7f3d00d9a700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ce006ace0 0x7f3ce006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.254+0000 7f3d00d9a700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ce006ace0 0x7f3ce006d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc15f220 tx=0x7f3cec009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.258+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3cf8053b80 con 0x7f3cfc0b5c90 2023-07-06T20:13:58.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:58.465+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3cfc0b6bd0 con 0x7f3cfc0b5c90 2023-07-06T20:13:59.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.001+0000 7f3cf1ffb700 1 -- 172.21.15.27:0/1234391910 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3cf8017020 con 0x7f3cfc0b5c90 2023-07-06T20:13:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.010+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ce006ace0 msgr2=0x7f3ce006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.010+0000 7f3d037ff700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ce006ace0 0x7f3ce006d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc15f220 tx=0x7f3cec009210 comp rx=0 tx=0).stop 2023-07-06T20:13:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.010+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 msgr2=0x7f3cfc152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.010+0000 7f3d037ff700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 0x7f3cfc152e90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c8c0 tx=0x7f3cf800cbd0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.016+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 shutdown_connections 2023-07-06T20:13:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.016+0000 7f3d037ff700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ce006ace0 0x7f3ce006d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.016+0000 7f3d037ff700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cfc15fca0 0x7f3cfc1545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.016+0000 7f3d037ff700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cfc0b5c90 0x7f3cfc152e90 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c8c0 tx=0x7f3cf800cbd0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.016+0000 7f3d037ff700 1 --2- 172.21.15.27:0/1234391910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cfc0b6110 0x7f3cfc0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.016+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 >> 172.21.15.27:0/1234391910 conn(0x7f3cfc06c150 msgr2=0x7f3cfc0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:59.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.020+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 shutdown_connections 2023-07-06T20:13:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.021+0000 7f3d037ff700 1 -- 172.21.15.27:0/1234391910 wait complete. 2023-07-06T20:13:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-07-06T20:13:59.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T20:13:59.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.251+0000 7f40dcbff700 1 Processor -- start 2023-07-06T20:13:59.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.252+0000 7f40dcbff700 1 -- start start 2023-07-06T20:13:59.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40dcbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40dcbff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40d80b5c90 0x7f40d80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40dcbff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40d80b6100 0x7f40d80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40dcbff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d80cf780 con 0x7f40d80b6100 2023-07-06T20:13:59.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40dcbff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d80cd490 con 0x7f40d80b5c90 2023-07-06T20:13:59.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40dcbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80cd770 con 0x7f40d815fca0 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40d6d9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40d80b6100 0x7f40d80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40d6d9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40d80b6100 0x7f40d80d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46608/0 (socket says 172.21.15.27:46608) 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40d6d9d700 1 -- 172.21.15.27:0/2445891420 learned_addr learned my addr 172.21.15.27:0/2445891420 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40d659c700 1 --2- 172.21.15.27:0/2445891420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40cffff700 1 --2- 172.21.15.27:0/2445891420 >> v2:172.21.15.153:3300/0 conn(0x7f40d80b5c90 0x7f40d80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40d6d9d700 1 -- 172.21.15.27:0/2445891420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 msgr2=0x7f40d80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.253+0000 7f40d6d9d700 1 --2- 172.21.15.27:0/2445891420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d6d9d700 1 -- 172.21.15.27:0/2445891420 >> v2:172.21.15.153:3300/0 conn(0x7f40d80b5c90 msgr2=0x7f40d80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d6d9d700 1 --2- 172.21.15.27:0/2445891420 >> v2:172.21.15.153:3300/0 conn(0x7f40d80b5c90 0x7f40d80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d6d9d700 1 -- 172.21.15.27:0/2445891420 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d80cda50 con 0x7f40d80b6100 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d6d9d700 1 --2- 172.21.15.27:0/2445891420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b6100 0x7f40d80d3280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f40d806cda0 tx=0x7f40d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f9efb5a9ebe36e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d5d9b700 1 -- 172.21.15.27:0/2445891420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000e070 con 0x7f40d80b6100 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d5d9b700 1 -- 172.21.15.27:0/2445891420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d000b9c0 con 0x7f40d80b6100 2023-07-06T20:13:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40d5d9b700 1 -- 172.21.15.27:0/2445891420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000bd70 con 0x7f40d80b6100 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40dcbff700 1 -- 172.21.15.27:0/2445891420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b6100 msgr2=0x7f40d80d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.254+0000 7f40dcbff700 1 --2- 172.21.15.27:0/2445891420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b6100 0x7f40d80d3280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f40d806cda0 tx=0x7f40d000b330 comp rx=0 tx=0).stop 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 -- 172.21.15.27:0/2445891420 shutdown_connections 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 --2- 172.21.15.27:0/2445891420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b6100 0x7f40d80d3280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 --2- 172.21.15.27:0/2445891420 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 --2- 172.21.15.27:0/2445891420 >> v2:172.21.15.153:3300/0 conn(0x7f40d80b5c90 0x7f40d80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 -- 172.21.15.27:0/2445891420 >> 172.21.15.27:0/2445891420 conn(0x7f40d806c150 msgr2=0x7f40d806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 -- 172.21.15.27:0/2445891420 shutdown_connections 2023-07-06T20:13:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.255+0000 7f40dcbff700 1 -- 172.21.15.27:0/2445891420 wait complete. 2023-07-06T20:13:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.256+0000 7f40dcbff700 1 Processor -- start 2023-07-06T20:13:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.257+0000 7f40dcbff700 1 -- start start 2023-07-06T20:13:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.257+0000 7f40dcbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 0x7f40d814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40dcbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80b6100 0x7f40d80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40dcbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40dcbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d815be50 con 0x7f40d80b5c90 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40dcbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80b49c0 con 0x7f40d80b6100 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40dcbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81e7e10 con 0x7f40d815fca0 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 0x7f40d814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 0x7f40d814a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46620/0 (socket says 172.21.15.27:46620) 2023-07-06T20:13:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 -- 172.21.15.27:0/1948012743 learned_addr learned my addr 172.21.15.27:0/1948012743 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40cffff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80b6100 0x7f40d80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d6d9d700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 -- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 msgr2=0x7f40d80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 -- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80b6100 msgr2=0x7f40d80b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80b6100 0x7f40d80b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.258+0000 7f40d659c700 1 -- 172.21.15.27:0/1948012743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d0009b30 con 0x7f40d80b5c90 2023-07-06T20:13:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.259+0000 7f40d659c700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 0x7f40d814a1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e810 tx=0x7f40c800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.259+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c800c070 con 0x7f40d80b5c90 2023-07-06T20:13:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.259+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8013610 con 0x7f40d80b5c90 2023-07-06T20:13:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.259+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d814a700 con 0x7f40d80b5c90 2023-07-06T20:13:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.259+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80139c0 con 0x7f40d80b5c90 2023-07-06T20:13:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.259+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d80b4120 con 0x7f40d80b5c90 2023-07-06T20:13:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.260+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d80008d0 con 0x7f40d80b5c90 2023-07-06T20:13:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.261+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40c8013b20 con 0x7f40d80b5c90 2023-07-06T20:13:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.262+0000 7f40ceffd700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f40b806ace0 0x7f40b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.262+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f40c80891f0 con 0x7f40d80b5c90 2023-07-06T20:13:59.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.263+0000 7f40cffff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f40b806ace0 0x7f40b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:59.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.263+0000 7f40cffff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f40b806ace0 0x7f40b806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40d80d2280 tx=0x7f40c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:59.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.267+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f40c8054270 con 0x7f40d80b5c90 2023-07-06T20:13:59.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:13:59.517+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f40d815bf90 con 0x7f40d80b5c90 2023-07-06T20:14:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.028+0000 7f40ceffd700 1 -- 172.21.15.27:0/1948012743 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f40c8057890 con 0x7f40d80b5c90 2023-07-06T20:14:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.033+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f40b806ace0 msgr2=0x7f40b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.033+0000 7f40dcbff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f40b806ace0 0x7f40b806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40d80d2280 tx=0x7f40c0008040 comp rx=0 tx=0).stop 2023-07-06T20:14:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.033+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 msgr2=0x7f40d814a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.033+0000 7f40dcbff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 0x7f40d814a1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e810 tx=0x7f40c800eb20 comp rx=0 tx=0).stop 2023-07-06T20:14:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.036+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 shutdown_connections 2023-07-06T20:14:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.036+0000 7f40dcbff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f40b806ace0 0x7f40b806d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.036+0000 7f40dcbff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40d815fca0 0x7f40d80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.036+0000 7f40dcbff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80b5c90 0x7f40d814a1c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.036+0000 7f40dcbff700 1 --2- 172.21.15.27:0/1948012743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80b6100 0x7f40d80b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.036+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 >> 172.21.15.27:0/1948012743 conn(0x7f40d806c150 msgr2=0x7f40d80d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:00.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.043+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 shutdown_connections 2023-07-06T20:14:00.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.045+0000 7f40dcbff700 1 -- 172.21.15.27:0/1948012743 wait complete. 2023-07-06T20:14:00.055 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 3 and data pool 4 2023-07-06T20:14:00.080 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.282+0000 7f38d3bff700 1 Processor -- start 2023-07-06T20:14:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.282+0000 7f38d3bff700 1 -- start start 2023-07-06T20:14:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.283+0000 7f38d3bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f38cc15fca0 0x7f38cc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.283+0000 7f38d3bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0cf780 con 0x7f38cc0b6110 2023-07-06T20:14:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.283+0000 7f38d3bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0cd490 con 0x7f38cc0b5c90 2023-07-06T20:14:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.283+0000 7f38d3bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0cd770 con 0x7f38cc15fca0 2023-07-06T20:14:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d219c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f38cc0b6110 0x7f38cc0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46058/0 (socket says 172.21.15.27:46058) 2023-07-06T20:14:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d219c700 1 -- 172.21.15.27:0/1226283018 learned_addr learned my addr 172.21.15.27:0/1226283018 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d199b700 1 --2- 172.21.15.27:0/1226283018 >> v2:172.21.15.153:3300/0 conn(0x7f38cc15fca0 0x7f38cc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009858143 0 0) 0x7f38cc0cf780 con 0x7f38cc0b6110 2023-07-06T20:14:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b80035b0 con 0x7f38cc0b6110 2023-07-06T20:14:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253875165 0 0) 0x7f38cc0cd490 con 0x7f38cc0b5c90 2023-07-06T20:14:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38cc0cf780 con 0x7f38cc0b5c90 2023-07-06T20:14:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d199b700 1 -- 172.21.15.27:0/1226283018 >> v1:172.21.15.153:6789/0 conn(0x7f38cc0b5c90 legacy=0x7f38cc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d199b700 1 -- 172.21.15.27:0/1226283018 >> v1:172.21.15.27:6789/0 conn(0x7f38cc0b6110 legacy=0x7f38cc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4151043305 0 0) 0x7f38b80035b0 con 0x7f38cc0b6110 2023-07-06T20:14:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.284+0000 7f38d199b700 1 -- 172.21.15.27:0/1226283018 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0cda50 con 0x7f38cc15fca0 2023-07-06T20:14:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.285+0000 7f38d199b700 1 --2- 172.21.15.27:0/1226283018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc0d1420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009840 tx=0x7f38c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c45d80469e2dd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.285+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40019f0 con 0x7f38cc15fca0 2023-07-06T20:14:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.285+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8004030 con 0x7f38cc15fca0 2023-07-06T20:14:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.285+0000 7f38d0999700 1 -- 172.21.15.27:0/1226283018 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80043c0 con 0x7f38cc15fca0 2023-07-06T20:14:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 -- 172.21.15.27:0/1226283018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 msgr2=0x7f38cc0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 --2- 172.21.15.27:0/1226283018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc0d1420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009840 tx=0x7f38c8009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 -- 172.21.15.27:0/1226283018 shutdown_connections 2023-07-06T20:14:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 --2- 172.21.15.27:0/1226283018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc0d1420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 -- 172.21.15.27:0/1226283018 >> 172.21.15.27:0/1226283018 conn(0x7f38cc06c150 msgr2=0x7f38cc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 -- 172.21.15.27:0/1226283018 shutdown_connections 2023-07-06T20:14:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.286+0000 7f38d3bff700 1 -- 172.21.15.27:0/1226283018 wait complete. 2023-07-06T20:14:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.288+0000 7f38d3bff700 1 Processor -- start 2023-07-06T20:14:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.288+0000 7f38d3bff700 1 -- start start 2023-07-06T20:14:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d3bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 0x7f38cc0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d3bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38cc0b6110 0x7f38cc1f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d3bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d3bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc155fa0 con 0x7f38cc0b5c90 2023-07-06T20:14:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d3bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc155110 con 0x7f38cc15fca0 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d3bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0cc5d0 con 0x7f38cc0b6110 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 0x7f38cc0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 0x7f38cc0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46628/0 (socket says 172.21.15.27:46628) 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 -- 172.21.15.27:0/873976650 learned_addr learned my addr 172.21.15.27:0/873976650 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d119a700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38cc0b6110 0x7f38cc1f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d219c700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 -- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38cc0b6110 msgr2=0x7f38cc1f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38cc0b6110 0x7f38cc1f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 -- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 msgr2=0x7f38cc158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.289+0000 7f38d199b700 1 -- 172.21.15.27:0/873976650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80094a0 con 0x7f38cc0b5c90 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.290+0000 7f38d199b700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 0x7f38cc0cc090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38c800a010 tx=0x7f38c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.290+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003db0 con 0x7f38cc0b5c90 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.290+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c801f070 con 0x7f38cc0b5c90 2023-07-06T20:14:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.290+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8011870 con 0x7f38cc0b5c90 2023-07-06T20:14:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.290+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc155320 con 0x7f38cc0b5c90 2023-07-06T20:14:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.290+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc1558e0 con 0x7f38cc0b5c90 2023-07-06T20:14:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.291+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc0008d0 con 0x7f38cc0b5c90 2023-07-06T20:14:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.292+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38c8021030 con 0x7f38cc0b5c90 2023-07-06T20:14:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.293+0000 7f38c27fc700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38b006ace0 0x7f38b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.293+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38c808a8a0 con 0x7f38cc0b5c90 2023-07-06T20:14:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.294+0000 7f38d119a700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38b006ace0 0x7f38b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.294+0000 7f38d119a700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38b006ace0 0x7f38b006d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0b45f0 tx=0x7f38bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:00.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.298+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38c8055940 con 0x7f38cc0b5c90 2023-07-06T20:14:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.505+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38cc0b4340 con 0x7f38cc0b5c90 2023-07-06T20:14:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.506+0000 7f38c27fc700 1 -- 172.21.15.27:0/873976650 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f38c8058f60 con 0x7f38cc0b5c90 2023-07-06T20:14:00.508 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:00.509 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:59.519294+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:00.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38b006ace0 msgr2=0x7f38b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38b006ace0 0x7f38b006d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0b45f0 tx=0x7f38bc009040 comp rx=0 tx=0).stop 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 msgr2=0x7f38cc0cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 0x7f38cc0cc090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38c800a010 tx=0x7f38c8004750 comp rx=0 tx=0).stop 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 shutdown_connections 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38b006ace0 0x7f38b006d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38cc15fca0 0x7f38cc158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38cc0b5c90 0x7f38cc0cc090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 --2- 172.21.15.27:0/873976650 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38cc0b6110 0x7f38cc1f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 >> 172.21.15.27:0/873976650 conn(0x7f38cc06c150 msgr2=0x7f38cc0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 shutdown_connections 2023-07-06T20:14:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.509+0000 7f38d3bff700 1 -- 172.21.15.27:0/873976650 wait complete. 2023-07-06T20:14:00.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.730+0000 7f6234bff700 1 Processor -- start 2023-07-06T20:14:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.730+0000 7f6234bff700 1 -- start start 2023-07-06T20:14:00.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.731+0000 7f6234bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f623015fca0 0x7f62300d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.731+0000 7f6234bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.731+0000 7f6234bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62300b60e0 0x7f62300d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.731+0000 7f6234bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300cf780 con 0x7f623015fca0 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.731+0000 7f6234bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300cd490 con 0x7f62300b60e0 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.731+0000 7f6234bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300cd770 con 0x7f62300b5c90 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f623015fca0 0x7f62300d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f623015fca0 0x7f62300d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46634/0 (socket says 172.21.15.27:46634) 2023-07-06T20:14:00.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 -- 172.21.15.27:0/335271052 learned_addr learned my addr 172.21.15.27:0/335271052 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:00.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f6227fff700 1 --2- 172.21.15.27:0/335271052 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622ed9d700 1 --2- 172.21.15.27:0/335271052 >> v2:172.21.15.153:3300/0 conn(0x7f62300b60e0 0x7f62300d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 -- 172.21.15.27:0/335271052 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 msgr2=0x7f62300d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:00.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 --2- 172.21.15.27:0/335271052 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 -- 172.21.15.27:0/335271052 >> v2:172.21.15.153:3300/0 conn(0x7f62300b60e0 msgr2=0x7f62300d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:00.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 --2- 172.21.15.27:0/335271052 >> v2:172.21.15.153:3300/0 conn(0x7f62300b60e0 0x7f62300d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622e59c700 1 -- 172.21.15.27:0/335271052 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300cda50 con 0x7f623015fca0 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.732+0000 7f622ed9d700 1 --2- 172.21.15.27:0/335271052 >> v2:172.21.15.153:3300/0 conn(0x7f62300b60e0 0x7f62300d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f622e59c700 1 --2- 172.21.15.27:0/335271052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623015fca0 0x7f62300d1420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62300b8f80 tx=0x7f6220009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe91a5d41df98430 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f622dd9b700 1 -- 172.21.15.27:0/335271052 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000b870 con 0x7f623015fca0 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f622dd9b700 1 -- 172.21.15.27:0/335271052 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622000b9d0 con 0x7f623015fca0 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f622dd9b700 1 -- 172.21.15.27:0/335271052 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000bd60 con 0x7f623015fca0 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 -- 172.21.15.27:0/335271052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623015fca0 msgr2=0x7f62300d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 --2- 172.21.15.27:0/335271052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623015fca0 0x7f62300d1420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62300b8f80 tx=0x7f6220009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 -- 172.21.15.27:0/335271052 shutdown_connections 2023-07-06T20:14:00.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 --2- 172.21.15.27:0/335271052 >> v2:172.21.15.153:3300/0 conn(0x7f62300b60e0 0x7f62300d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 --2- 172.21.15.27:0/335271052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623015fca0 0x7f62300d1420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 --2- 172.21.15.27:0/335271052 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.733+0000 7f6234bff700 1 -- 172.21.15.27:0/335271052 >> 172.21.15.27:0/335271052 conn(0x7f623006c150 msgr2=0x7f623006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:00.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.734+0000 7f6234bff700 1 -- 172.21.15.27:0/335271052 shutdown_connections 2023-07-06T20:14:00.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.734+0000 7f6234bff700 1 -- 172.21.15.27:0/335271052 wait complete. 2023-07-06T20:14:00.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.735+0000 7f6234bff700 1 Processor -- start 2023-07-06T20:14:00.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.736+0000 7f6234bff700 1 -- start start 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.736+0000 7f6234bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6234bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 0x7f62300cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6234bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623015fca0 0x7f62300b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6234bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300a98c0 con 0x7f62300b60e0 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6234bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623015c8d0 con 0x7f623015fca0 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6234bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623015b830 con 0x7f62300b5c90 2023-07-06T20:14:00.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 0x7f62300cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f622e59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 0x7f62300cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46648/0 (socket says 172.21.15.27:46648) 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 -- 172.21.15.27:0/1054631643 learned_addr learned my addr 172.21.15.27:0/1054631643 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f622ed9d700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623015fca0 0x7f62300b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 -- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 msgr2=0x7f62300cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 -- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623015fca0 msgr2=0x7f62300b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623015fca0 0x7f62300b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.737+0000 7f6227fff700 1 -- 172.21.15.27:0/1054631643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6220009550 con 0x7f62300b60e0 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.738+0000 7f6227fff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 0x7f62300cbbd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f621800e810 tx=0x7f621800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.738+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800c070 con 0x7f62300b60e0 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.738+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218013610 con 0x7f62300b60e0 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.738+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180139a0 con 0x7f62300b60e0 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.738+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623015bb10 con 0x7f62300b60e0 2023-07-06T20:14:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.738+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623015c0d0 con 0x7f62300b60e0 2023-07-06T20:14:00.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.739+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62300008d0 con 0x7f62300b60e0 2023-07-06T20:14:00.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.740+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6218013b00 con 0x7f62300b60e0 2023-07-06T20:14:00.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.741+0000 7f6226ffd700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f621006ace0 0x7f621006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:00.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.741+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6218089160 con 0x7f62300b60e0 2023-07-06T20:14:00.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.741+0000 7f622e59c700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f621006ace0 0x7f621006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:00.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.742+0000 7f622e59c700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f621006ace0 0x7f621006d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6220014040 tx=0x7f6220008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:00.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.745+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6218054200 con 0x7f62300b60e0 2023-07-06T20:14:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.952+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f623015caf0 con 0x7f62300b60e0 2023-07-06T20:14:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.953+0000 7f6226ffd700 1 -- 172.21.15.27:0/1054631643 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f6218009440 con 0x7f62300b60e0 2023-07-06T20:14:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:00.956 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:13:59.519294+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.955+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f621006ace0 msgr2=0x7f621006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f621006ace0 0x7f621006d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6220014040 tx=0x7f6220008030 comp rx=0 tx=0).stop 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 msgr2=0x7f62300cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 0x7f62300cbbd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f621800e810 tx=0x7f621800eb20 comp rx=0 tx=0).stop 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 shutdown_connections 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f621006ace0 0x7f621006d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f623015fca0 0x7f62300b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62300b5c90 0x7f62300cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 --2- 172.21.15.27:0/1054631643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62300b60e0 0x7f62300cbbd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 >> 172.21.15.27:0/1054631643 conn(0x7f623006c150 msgr2=0x7f62300cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 shutdown_connections 2023-07-06T20:14:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:00.956+0000 7f6234bff700 1 -- 172.21.15.27:0/1054631643 wait complete. 2023-07-06T20:14:00.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T20:14:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.181+0000 7f85d01ff700 1 Processor -- start 2023-07-06T20:14:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.182+0000 7f85d01ff700 1 -- start start 2023-07-06T20:14:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85d01ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f85c80b5c90 0x7f85c80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85d01ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80cf780 con 0x7f85c815fca0 2023-07-06T20:14:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85d01ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80cd490 con 0x7f85c80b6110 2023-07-06T20:14:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85d01ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80cd770 con 0x7f85c80b5c90 2023-07-06T20:14:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85cdf9b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f85c815fca0 0x7f85c80d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46072/0 (socket says 172.21.15.27:46072) 2023-07-06T20:14:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85cdf9b700 1 -- 172.21.15.27:0/1927046049 learned_addr learned my addr 172.21.15.27:0/1927046049 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.183+0000 7f85cd79a700 1 --2- 172.21.15.27:0/1927046049 >> v2:172.21.15.153:3300/0 conn(0x7f85c80b5c90 0x7f85c80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85ccf99700 1 -- 172.21.15.27:0/1927046049 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923410604 0 0) 0x7f85c80cf780 con 0x7f85c815fca0 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85ccf99700 1 -- 172.21.15.27:0/1927046049 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b40035b0 con 0x7f85c815fca0 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85cd79a700 1 -- 172.21.15.27:0/1927046049 >> v1:172.21.15.153:6789/0 conn(0x7f85c80b6110 legacy=0x7f85c80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85cd79a700 1 -- 172.21.15.27:0/1927046049 >> v1:172.21.15.27:6789/0 conn(0x7f85c815fca0 legacy=0x7f85c80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85cd79a700 1 -- 172.21.15.27:0/1927046049 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80cda50 con 0x7f85c80b5c90 2023-07-06T20:14:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85cd79a700 1 --2- 172.21.15.27:0/1927046049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c80d2d40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009840 tx=0x7f85b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8d79f0124b6c43 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85ccf99700 1 -- 172.21.15.27:0/1927046049 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8011830 con 0x7f85c80b5c90 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85ccf99700 1 -- 172.21.15.27:0/1927046049 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8011990 con 0x7f85c80b5c90 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.184+0000 7f85ccf99700 1 -- 172.21.15.27:0/1927046049 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8011d20 con 0x7f85c80b5c90 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 -- 172.21.15.27:0/1927046049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 msgr2=0x7f85c80d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 --2- 172.21.15.27:0/1927046049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c80d2d40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009840 tx=0x7f85b8009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 -- 172.21.15.27:0/1927046049 shutdown_connections 2023-07-06T20:14:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 --2- 172.21.15.27:0/1927046049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c80d2d40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 -- 172.21.15.27:0/1927046049 >> 172.21.15.27:0/1927046049 conn(0x7f85c806c150 msgr2=0x7f85c806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 -- 172.21.15.27:0/1927046049 shutdown_connections 2023-07-06T20:14:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.185+0000 7f85d01ff700 1 -- 172.21.15.27:0/1927046049 wait complete. 2023-07-06T20:14:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.187+0000 7f85d01ff700 1 Processor -- start 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.187+0000 7f85d01ff700 1 -- start start 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85d01ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c8157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85d01ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 0x7f85c80c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85d01ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85c815fca0 0x7f85c80b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85d01ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80c8880 con 0x7f85c80b6110 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85d01ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81e61c0 con 0x7f85c80b5c90 2023-07-06T20:14:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85d01ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8157a60 con 0x7f85c815fca0 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85cd79a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 0x7f85c80c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85cd79a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 0x7f85c80c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46664/0 (socket says 172.21.15.27:46664) 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85cd79a700 1 -- 172.21.15.27:0/2637324981 learned_addr learned my addr 172.21.15.27:0/2637324981 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85cdf9b700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c8157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.188+0000 7f85ce79c700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85c815fca0 0x7f85c80b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cd79a700 1 -- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85c815fca0 msgr2=0x7f85c80b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cd79a700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85c815fca0 0x7f85c80b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cd79a700 1 -- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 msgr2=0x7f85c8157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cd79a700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c8157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cd79a700 1 -- 172.21.15.27:0/2637324981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80094a0 con 0x7f85c80b6110 2023-07-06T20:14:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cdf9b700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c8157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85cd79a700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 0x7f85c80c7320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009fd0 tx=0x7f85b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80038a0 con 0x7f85c80b6110 2023-07-06T20:14:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b801f070 con 0x7f85c80b6110 2023-07-06T20:14:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c80c7c70 con 0x7f85c80b6110 2023-07-06T20:14:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8003c40 con 0x7f85c80b6110 2023-07-06T20:14:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.189+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80c8230 con 0x7f85c80b6110 2023-07-06T20:14:01.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.190+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c81e63c0 con 0x7f85c80b6110 2023-07-06T20:14:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.191+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85b8026480 con 0x7f85c80b6110 2023-07-06T20:14:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.193+0000 7f85be7fc700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f85ac06ace0 0x7f85ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.193+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85b8021070 con 0x7f85c80b6110 2023-07-06T20:14:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.193+0000 7f85cdf9b700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f85ac06ace0 0x7f85ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:01.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.194+0000 7f85cdf9b700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f85ac06ace0 0x7f85ac06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85c40011e0 tx=0x7f85c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.198+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85c81e63c0 con 0x7f85c80b6110 2023-07-06T20:14:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:01.452+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f85c81e6380 con 0x7f85c80b6110 2023-07-06T20:14:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.050+0000 7f85be7fc700 1 -- 172.21.15.27:0/2637324981 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f85b8055e50 con 0x7f85c80b6110 2023-07-06T20:14:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f85ac06ace0 msgr2=0x7f85ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f85ac06ace0 0x7f85ac06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85c40011e0 tx=0x7f85c4009040 comp rx=0 tx=0).stop 2023-07-06T20:14:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 msgr2=0x7f85c80c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 0x7f85c80c7320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009fd0 tx=0x7f85b8004750 comp rx=0 tx=0).stop 2023-07-06T20:14:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 shutdown_connections 2023-07-06T20:14:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f85ac06ace0 0x7f85ac06d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85c815fca0 0x7f85c80b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85c80b5c90 0x7f85c8157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 --2- 172.21.15.27:0/2637324981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85c80b6110 0x7f85c80c7320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 >> 172.21.15.27:0/2637324981 conn(0x7f85c806c150 msgr2=0x7f85c80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 shutdown_connections 2023-07-06T20:14:02.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.053+0000 7f85d01ff700 1 -- 172.21.15.27:0/2637324981 wait complete. 2023-07-06T20:14:02.074 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T20:14:02.075 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T20:14:02.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.281+0000 7fbfb57ff700 1 Processor -- start 2023-07-06T20:14:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.281+0000 7fbfb57ff700 1 -- start start 2023-07-06T20:14:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfb57ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfb57ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbfb00b5c90 0x7fbfb00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfb57ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb00cf780 con 0x7fbfb00b6100 2023-07-06T20:14:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfb57ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb00cd490 con 0x7fbfb00b5c90 2023-07-06T20:14:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfb57ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb00cd770 con 0x7fbfb015fca0 2023-07-06T20:14:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfaf7fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbfb00b6100 0x7fbfb00cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46048/0 (socket says 172.21.15.27:46048) 2023-07-06T20:14:02.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfaf7fe700 1 -- 172.21.15.27:0/3021622398 learned_addr learned my addr 172.21.15.27:0/3021622398 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfae7fc700 1 --2- 172.21.15.27:0/3021622398 >> v2:172.21.15.153:3300/0 conn(0x7fbfb00b5c90 0x7fbfb00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.282+0000 7fbfaeffd700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfadffb700 1 -- 172.21.15.27:0/3021622398 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482298108 0 0) 0x7fbfb00cf780 con 0x7fbfb00b6100 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfadffb700 1 -- 172.21.15.27:0/3021622398 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf980035b0 con 0x7fbfb00b6100 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfae7fc700 1 -- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 msgr2=0x7fbfb00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfae7fc700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfae7fc700 1 -- 172.21.15.27:0/3021622398 >> v1:172.21.15.27:6789/0 conn(0x7fbfb00b6100 legacy=0x7fbfb00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfae7fc700 1 -- 172.21.15.27:0/3021622398 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00cda50 con 0x7fbfb00b5c90 2023-07-06T20:14:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfaeffd700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.283+0000 7fbfae7fc700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00d2d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009840 tx=0x7fbfa4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e28599fddc3ade server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfadffb700 1 -- 172.21.15.27:0/3021622398 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4011830 con 0x7fbfb00b5c90 2023-07-06T20:14:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfadffb700 1 -- 172.21.15.27:0/3021622398 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa4011990 con 0x7fbfb00b5c90 2023-07-06T20:14:02.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfadffb700 1 -- 172.21.15.27:0/3021622398 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4011d20 con 0x7fbfb00b5c90 2023-07-06T20:14:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 -- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b5c90 msgr2=0x7fbfb00d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00d2d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009840 tx=0x7fbfa4009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 -- 172.21.15.27:0/3021622398 shutdown_connections 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/3021622398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00d2d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 -- 172.21.15.27:0/3021622398 >> 172.21.15.27:0/3021622398 conn(0x7fbfb006c150 msgr2=0x7fbfb006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 -- 172.21.15.27:0/3021622398 shutdown_connections 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.284+0000 7fbfb57ff700 1 -- 172.21.15.27:0/3021622398 wait complete. 2023-07-06T20:14:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.286+0000 7fbfb57ff700 1 Processor -- start 2023-07-06T20:14:02.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.286+0000 7fbfb57ff700 1 -- start start 2023-07-06T20:14:02.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfb57ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfb57ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b6100 0x7fbfb015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfb57ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfb57ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb015b7c0 con 0x7fbfb00b5c90 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfb57ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb015b9a0 con 0x7fbfb00b6100 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfb57ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb015bc80 con 0x7fbfb015fca0 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfaeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfaeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59148/0 (socket says 172.21.15.27:59148) 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.287+0000 7fbfaeffd700 1 -- 172.21.15.27:0/2798776080 learned_addr learned my addr 172.21.15.27:0/2798776080 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfae7fc700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b6100 0x7fbfb015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfaeffd700 1 -- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 msgr2=0x7fbfb00d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfaeffd700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfaeffd700 1 -- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b6100 msgr2=0x7fbfb015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfaeffd700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b6100 0x7fbfb015ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfaeffd700 1 -- 172.21.15.27:0/2798776080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa00094a0 con 0x7fbfb00b5c90 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfaeffd700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00cab20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000ebe0 tx=0x7fbfa0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000c8b0 con 0x7fbfb00b5c90 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa000ca10 con 0x7fbfb00b5c90 2023-07-06T20:14:02.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000cda0 con 0x7fbfb00b5c90 2023-07-06T20:14:02.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa40094a0 con 0x7fbfb00b5c90 2023-07-06T20:14:02.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.288+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb015c240 con 0x7fbfb00b5c90 2023-07-06T20:14:02.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.289+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb0149b20 con 0x7fbfb00b5c90 2023-07-06T20:14:02.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.290+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfa0013470 con 0x7fbfb00b5c90 2023-07-06T20:14:02.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.291+0000 7fbf977fe700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf9c06ace0 0x7fbf9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.291+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfa00896c0 con 0x7fbfb00b5c90 2023-07-06T20:14:02.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.292+0000 7fbfae7fc700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf9c06ace0 0x7fbf9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.292+0000 7fbfae7fc700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf9c06ace0 0x7fbf9c06d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0157a80 tx=0x7fbfa4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:02.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.296+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbfa0054640 con 0x7fbfb00b5c90 2023-07-06T20:14:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:02.549+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbfb015c5b0 con 0x7fbfb00b5c90 2023-07-06T20:14:03.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.056+0000 7fbf977fe700 1 -- 172.21.15.27:0/2798776080 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbfa0057c60 con 0x7fbfb00b5c90 2023-07-06T20:14:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf9c06ace0 msgr2=0x7fbf9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf9c06ace0 0x7fbf9c06d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0157a80 tx=0x7fbfa4008030 comp rx=0 tx=0).stop 2023-07-06T20:14:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 msgr2=0x7fbfb00cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00cab20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000ebe0 tx=0x7fbfa0009e20 comp rx=0 tx=0).stop 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 shutdown_connections 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf9c06ace0 0x7fbf9c06d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbfb015fca0 0x7fbfb00d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb00b5c90 0x7fbfb00cab20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 --2- 172.21.15.27:0/2798776080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbfb00b6100 0x7fbfb015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 >> 172.21.15.27:0/2798776080 conn(0x7fbfb006c150 msgr2=0x7fbfb00b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:03.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.059+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 shutdown_connections 2023-07-06T20:14:03.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.060+0000 7fbfb57ff700 1 -- 172.21.15.27:0/2798776080 wait complete. 2023-07-06T20:14:03.081 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T20:14:03.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:14:03.182 INFO:tasks.ceph.mds.a.smithi027.stderr:2023-07-06T20:14:03.180+0000 7fb66b9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T20:14:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.287+0000 7f3f249ff700 1 Processor -- start 2023-07-06T20:14:03.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.287+0000 7f3f249ff700 1 -- start start 2023-07-06T20:14:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f249ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f200b5c90 0x7f3f200d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f249ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f200cf780 con 0x7f3f200b6110 2023-07-06T20:14:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f249ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f200cd490 con 0x7f3f2015fca0 2023-07-06T20:14:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f249ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f200cd770 con 0x7f3f200b5c90 2023-07-06T20:14:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f1dd9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f200b5c90 0x7f3f200d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f1dd9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f200b5c90 0x7f3f200d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59160/0 (socket says 172.21.15.27:59160) 2023-07-06T20:14:03.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f1dd9b700 1 -- 172.21.15.27:0/3345501914 learned_addr learned my addr 172.21.15.27:0/3345501914 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:03.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.289+0000 7f3f1ed9d700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3f200b6110 0x7f3f200cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46056/0 (socket says 172.21.15.27:46056) 2023-07-06T20:14:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1dd9b700 1 -- 172.21.15.27:0/3345501914 >> v1:172.21.15.153:6789/0 conn(0x7f3f2015fca0 legacy=0x7f3f200d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1dd9b700 1 -- 172.21.15.27:0/3345501914 >> v1:172.21.15.27:6789/0 conn(0x7f3f200b6110 legacy=0x7f3f200cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1dd9b700 1 -- 172.21.15.27:0/3345501914 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200cda50 con 0x7f3f200b5c90 2023-07-06T20:14:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1dd9b700 1 --2- 172.21.15.27:0/3345501914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f200d2d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098f0 tx=0x7f3f14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0161c71770c02b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1d59a700 1 -- 172.21.15.27:0/3345501914 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400b8a0 con 0x7f3f200b5c90 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1d59a700 1 -- 172.21.15.27:0/3345501914 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1400ba00 con 0x7f3f200b5c90 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.290+0000 7f3f1d59a700 1 -- 172.21.15.27:0/3345501914 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400bd90 con 0x7f3f200b5c90 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 -- 172.21.15.27:0/3345501914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 msgr2=0x7f3f200d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 --2- 172.21.15.27:0/3345501914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f200d2d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098f0 tx=0x7f3f14009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 -- 172.21.15.27:0/3345501914 shutdown_connections 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 --2- 172.21.15.27:0/3345501914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f200d2d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 -- 172.21.15.27:0/3345501914 >> 172.21.15.27:0/3345501914 conn(0x7f3f2006c150 msgr2=0x7f3f2006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:03.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 -- 172.21.15.27:0/3345501914 shutdown_connections 2023-07-06T20:14:03.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.291+0000 7f3f249ff700 1 -- 172.21.15.27:0/3345501914 wait complete. 2023-07-06T20:14:03.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.293+0000 7f3f249ff700 1 Processor -- start 2023-07-06T20:14:03.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.293+0000 7f3f249ff700 1 -- start start 2023-07-06T20:14:03.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f249ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f20157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f249ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f200b6110 0x7f3f200c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f249ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2015fca0 0x7f3f200c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f249ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200cafe0 con 0x7f3f200b5c90 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f249ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f201e5ee0 con 0x7f3f200b6110 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f249ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f201576a0 con 0x7f3f2015fca0 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f20157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f20157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59168/0 (socket says 172.21.15.27:59168) 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 -- 172.21.15.27:0/2787413405 learned_addr learned my addr 172.21.15.27:0/2787413405 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:03.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1dd9b700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f200b6110 0x7f3f200c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1ed9d700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2015fca0 0x7f3f200c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 -- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2015fca0 msgr2=0x7f3f200c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2015fca0 0x7f3f200c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 -- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f200b6110 msgr2=0x7f3f200c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f200b6110 0x7f3f200c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1e59c700 1 -- 172.21.15.27:0/2787413405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14009550 con 0x7f3f200b5c90 2023-07-06T20:14:03.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.294+0000 7f3f1ed9d700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2015fca0 0x7f3f200c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:03.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.295+0000 7f3f1e59c700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f20157160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c480 tx=0x7f3f1000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:03.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.295+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f100052d0 con 0x7f3f200b5c90 2023-07-06T20:14:03.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.295+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10005430 con 0x7f3f200b5c90 2023-07-06T20:14:03.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.295+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f200ca460 con 0x7f3f200b5c90 2023-07-06T20:14:03.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.295+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f100038c0 con 0x7f3f200b5c90 2023-07-06T20:14:03.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.295+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f200caa20 con 0x7f3f200b5c90 2023-07-06T20:14:03.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.296+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f200008d0 con 0x7f3f200b5c90 2023-07-06T20:14:03.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.297+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f10003a20 con 0x7f3f200b5c90 2023-07-06T20:14:03.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.298+0000 7f3f0effd700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f0006ace0 0x7f3f0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.298+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f10088820 con 0x7f3f200b5c90 2023-07-06T20:14:03.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.299+0000 7f3f1dd9b700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f0006ace0 0x7f3f0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.299+0000 7f3f1dd9b700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f0006ace0 0x7f3f0006d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009c40 tx=0x7f3f14008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:03.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.302+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f100538d0 con 0x7f3f200b5c90 2023-07-06T20:14:03.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.550+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f200738f0 con 0x7f3f200b5c90 2023-07-06T20:14:03.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.551+0000 7f3f0effd700 1 -- 172.21.15.27:0/2787413405 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f3f10056ef0 con 0x7f3f200b5c90 2023-07-06T20:14:03.552 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:03.553 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f0006ace0 msgr2=0x7f3f0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f0006ace0 0x7f3f0006d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009c40 tx=0x7f3f14008030 comp rx=0 tx=0).stop 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 msgr2=0x7f3f20157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f20157160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c480 tx=0x7f3f1000c790 comp rx=0 tx=0).stop 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 shutdown_connections 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f0006ace0 0x7f3f0006d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f2015fca0 0x7f3f200c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f200b5c90 0x7f3f20157160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 --2- 172.21.15.27:0/2787413405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f200b6110 0x7f3f200c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 >> 172.21.15.27:0/2787413405 conn(0x7f3f2006c150 msgr2=0x7f3f200d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 shutdown_connections 2023-07-06T20:14:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.553+0000 7f3f249ff700 1 -- 172.21.15.27:0/2787413405 wait complete. 2023-07-06T20:14:03.557 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:14:03.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.775+0000 7f2b1afff700 1 Processor -- start 2023-07-06T20:14:03.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.775+0000 7f2b1afff700 1 -- start start 2023-07-06T20:14:03.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.776+0000 7f2b1afff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b140b5c90 0x7f2b140d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.776+0000 7f2b1afff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b140cf780 con 0x7f2b1415fca0 2023-07-06T20:14:03.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.776+0000 7f2b1afff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b140cd490 con 0x7f2b140b6110 2023-07-06T20:14:03.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.776+0000 7f2b1afff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b140cd770 con 0x7f2b140b5c90 2023-07-06T20:14:03.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b140b5c90 0x7f2b140d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b140b5c90 0x7f2b140d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59178/0 (socket says 172.21.15.27:59178) 2023-07-06T20:14:03.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 -- 172.21.15.27:0/4095783477 learned_addr learned my addr 172.21.15.27:0/4095783477 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:03.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b18d9b700 1 --1- 172.21.15.27:0/4095783477 >> v1:172.21.15.27:6789/0 conn(0x7f2b1415fca0 0x7f2b140d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46062/0 (socket says 172.21.15.27:46062) 2023-07-06T20:14:03.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 -- 172.21.15.27:0/4095783477 >> v1:172.21.15.153:6789/0 conn(0x7f2b140b6110 legacy=0x7f2b140cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:03.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 -- 172.21.15.27:0/4095783477 >> v1:172.21.15.27:6789/0 conn(0x7f2b1415fca0 legacy=0x7f2b140d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 -- 172.21.15.27:0/4095783477 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140cda50 con 0x7f2b140b5c90 2023-07-06T20:14:03.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b137fe700 1 -- 172.21.15.27:0/4095783477 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648980525 0 0) 0x7f2b140cf780 con 0x7f2b1415fca0 2023-07-06T20:14:03.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.777+0000 7f2b13fff700 1 --2- 172.21.15.27:0/4095783477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b5c90 0x7f2b140d2d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b000098f0 tx=0x7f2b00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c5db0822471eaa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b137fe700 1 -- 172.21.15.27:0/4095783477 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000b8a0 con 0x7f2b140b5c90 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b137fe700 1 -- 172.21.15.27:0/4095783477 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0000ba00 con 0x7f2b140b5c90 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b137fe700 1 -- 172.21.15.27:0/4095783477 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0000bd90 con 0x7f2b140b5c90 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 -- 172.21.15.27:0/4095783477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b5c90 msgr2=0x7f2b140d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 --2- 172.21.15.27:0/4095783477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b5c90 0x7f2b140d2d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b000098f0 tx=0x7f2b00009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 -- 172.21.15.27:0/4095783477 shutdown_connections 2023-07-06T20:14:03.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 --2- 172.21.15.27:0/4095783477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b5c90 0x7f2b140d2d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 -- 172.21.15.27:0/4095783477 >> 172.21.15.27:0/4095783477 conn(0x7f2b1406c150 msgr2=0x7f2b1406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:03.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 -- 172.21.15.27:0/4095783477 shutdown_connections 2023-07-06T20:14:03.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.778+0000 7f2b1afff700 1 -- 172.21.15.27:0/4095783477 wait complete. 2023-07-06T20:14:03.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.780+0000 7f2b1afff700 1 Processor -- start 2023-07-06T20:14:03.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.780+0000 7f2b1afff700 1 -- start start 2023-07-06T20:14:03.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1afff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b140b5c90 0x7f2b1415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1afff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 0x7f2b141f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1afff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b1415fca0 0x7f2b141f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1afff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141f1b50 con 0x7f2b140b6110 2023-07-06T20:14:03.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1afff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141f60d0 con 0x7f2b1415fca0 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1afff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141f5b10 con 0x7f2b140b5c90 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b13fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 0x7f2b141f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b13fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 0x7f2b141f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59192/0 (socket says 172.21.15.27:59192) 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b13fff700 1 -- 172.21.15.27:0/1005065129 learned_addr learned my addr 172.21.15.27:0/1005065129 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b1959c700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b1415fca0 0x7f2b141f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.781+0000 7f2b18d9b700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b140b5c90 0x7f2b1415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b13fff700 1 -- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b140b5c90 msgr2=0x7f2b1415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b13fff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b140b5c90 0x7f2b1415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b13fff700 1 -- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b1415fca0 msgr2=0x7f2b141f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b13fff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b1415fca0 0x7f2b141f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b13fff700 1 -- 172.21.15.27:0/1005065129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00009550 con 0x7f2b140b6110 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b1959c700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b1415fca0 0x7f2b141f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b13fff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 0x7f2b141f2550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b1406d0f0 tx=0x7f2b00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00011a70 con 0x7f2b140b6110 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0001f070 con 0x7f2b140b6110 2023-07-06T20:14:03.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00011dd0 con 0x7f2b140b6110 2023-07-06T20:14:03.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b141f5df0 con 0x7f2b140b6110 2023-07-06T20:14:03.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.782+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b141f2030 con 0x7f2b140b6110 2023-07-06T20:14:03.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.783+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b140008d0 con 0x7f2b140b6110 2023-07-06T20:14:03.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.784+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b0001a430 con 0x7f2b140b6110 2023-07-06T20:14:03.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.785+0000 7f2b117fa700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2af806ace0 0x7f2af806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:03.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.786+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b00021070 con 0x7f2b140b6110 2023-07-06T20:14:03.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.786+0000 7f2b18d9b700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2af806ace0 0x7f2af806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:03.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.786+0000 7f2b18d9b700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2af806ace0 0x7f2af806d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b08000f60 tx=0x7f2b08007db0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:03.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.789+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b00056030 con 0x7f2b140b6110 2023-07-06T20:14:03.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.996+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b140b2c40 con 0x7f2b140b6110 2023-07-06T20:14:03.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:03.997+0000 7f2b117fa700 1 -- 172.21.15.27:0/1005065129 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f2b00015090 con 0x7f2b140b6110 2023-07-06T20:14:03.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:04.000 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:04.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2af806ace0 msgr2=0x7f2af806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2af806ace0 0x7f2af806d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b08000f60 tx=0x7f2b08007db0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 msgr2=0x7f2b141f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 0x7f2b141f2550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b1406d0f0 tx=0x7f2b00004750 comp rx=0 tx=0).stop 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 shutdown_connections 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2af806ace0 0x7f2af806d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b1415fca0 0x7f2b141f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b140b5c90 0x7f2b1415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 --2- 172.21.15.27:0/1005065129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b140b6110 0x7f2b141f2550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 >> 172.21.15.27:0/1005065129 conn(0x7f2b1406c150 msgr2=0x7f2b140d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 shutdown_connections 2023-07-06T20:14:04.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.000+0000 7f2b1afff700 1 -- 172.21.15.27:0/1005065129 wait complete. 2023-07-06T20:14:04.020 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T20:14:04.020 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:04.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.222+0000 7fdf141ff700 1 Processor -- start 2023-07-06T20:14:04.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.223+0000 7fdf141ff700 1 -- start start 2023-07-06T20:14:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf141ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 0x7fdf0c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf141ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0cf780 con 0x7fdf0c0b6100 2023-07-06T20:14:04.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf141ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0cd490 con 0x7fdf0c0b5c90 2023-07-06T20:14:04.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf141ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0cd770 con 0x7fdf0c15fca0 2023-07-06T20:14:04.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf1279c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdf0c0b6100 0x7fdf0c0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46066/0 (socket says 172.21.15.27:46066) 2023-07-06T20:14:04.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf1279c700 1 -- 172.21.15.27:0/1321244333 learned_addr learned my addr 172.21.15.27:0/1321244333 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf11f9b700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 0x7fdf0c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790309807 0 0) 0x7fdf0c0cf780 con 0x7fdf0c0b6100 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.224+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef80035b0 con 0x7fdf0c0b6100 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709422024 0 0) 0x7fdef80035b0 con 0x7fdf0c0b6100 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 msgr2=0x7fdf0c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 0x7fdf0c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 >> v1:172.21.15.153:6789/0 conn(0x7fdf0c0b5c90 legacy=0x7fdf0c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0cda50 con 0x7fdf0c0b6100 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7fdf040019f0 con 0x7fdf0c0b6100 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 >> v1:172.21.15.27:6789/0 conn(0x7fdf0c0b6100 legacy=0x7fdf0c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 0x7fdef8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8003f70 0x7fdef8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8004d90 0x7fdef8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0cd770 con 0x7fdef8003f70 2023-07-06T20:14:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef800bac0 con 0x7fdef8004d90 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef800bc80 con 0x7fdf0c15fca0 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf1179a700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8003f70 0x7fdef8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf11f9b700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 0x7fdef8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.225+0000 7fdf1279c700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8004d90 0x7fdef8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf1179a700 1 -- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 msgr2=0x7fdef8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf1179a700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c15fca0 0x7fdef8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf1179a700 1 -- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8004d90 msgr2=0x7fdef8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf1179a700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8004d90 0x7fdef8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf1179a700 1 -- 172.21.15.27:0/1321244333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80035b0 con 0x7fdef8003f70 2023-07-06T20:14:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf1179a700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8003f70 0x7fdef8004850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdefc007fd0 tx=0x7fdef8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b26375a6760b3c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:04.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00c660 con 0x7fdef8003f70 2023-07-06T20:14:04.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc00c7c0 con 0x7fdef8003f70 2023-07-06T20:14:04.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.226+0000 7fdf10f99700 1 -- 172.21.15.27:0/1321244333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00cb70 con 0x7fdef8003f70 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf141ff700 1 -- 172.21.15.27:0/1321244333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8003f70 msgr2=0x7fdef8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf141ff700 1 --2- 172.21.15.27:0/1321244333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8003f70 0x7fdef8004850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdefc007fd0 tx=0x7fdef8005880 comp rx=0 tx=0).stop 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf1279c700 1 -- 172.21.15.27:0/1321244333 reap_dead start 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf141ff700 1 -- 172.21.15.27:0/1321244333 shutdown_connections 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf141ff700 1 -- 172.21.15.27:0/1321244333 >> 172.21.15.27:0/1321244333 conn(0x7fdf0c06c150 msgr2=0x7fdf0c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf141ff700 1 -- 172.21.15.27:0/1321244333 shutdown_connections 2023-07-06T20:14:04.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.227+0000 7fdf141ff700 1 -- 172.21.15.27:0/1321244333 wait complete. 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.229+0000 7fdf141ff700 1 Processor -- start 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.229+0000 7fdf141ff700 1 -- start start 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf141ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8003f70 0x7fdf0c14a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf141ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 0x7fdf0c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf141ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c0ca030 0x7fdf0c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf141ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0b3ab0 con 0x7fdf0c0b3100 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf141ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0cadd0 con 0x7fdef8003f70 2023-07-06T20:14:04.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf141ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0cb0b0 con 0x7fdf0c0ca030 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 0x7fdf0c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 0x7fdf0c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59212/0 (socket says 172.21.15.27:59212) 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 -- 172.21.15.27:0/2387910733 learned_addr learned my addr 172.21.15.27:0/2387910733 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf11f9b700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8003f70 0x7fdf0c14a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1279c700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c0ca030 0x7fdf0c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 -- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c0ca030 msgr2=0x7fdf0c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c0ca030 0x7fdf0c0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 -- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8003f70 msgr2=0x7fdf0c14a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8003f70 0x7fdf0c14a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 -- 172.21.15.27:0/2387910733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0cb390 con 0x7fdf0c0b3100 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.230+0000 7fdf1179a700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 0x7fdf0c0b3570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdef80047b0 tx=0x7fdefc001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.231+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00cb30 con 0x7fdf0c0b3100 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.231+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc021070 con 0x7fdf0c0b3100 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.231+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c0cb670 con 0x7fdf0c0b3100 2023-07-06T20:14:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.231+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc001330 con 0x7fdf0c0b3100 2023-07-06T20:14:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.231+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c0ca9e0 con 0x7fdf0c0b3100 2023-07-06T20:14:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.232+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c0008d0 con 0x7fdf0c0b3100 2023-07-06T20:14:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.233+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdefc023030 con 0x7fdf0c0b3100 2023-07-06T20:14:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.234+0000 7fdf027fc700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdef006ad10 0x7fdef006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.235+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdefc08a0e0 con 0x7fdf0c0b3100 2023-07-06T20:14:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.235+0000 7fdf11f9b700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdef006ad10 0x7fdef006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.236+0000 7fdf11f9b700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdef006ad10 0x7fdef006d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009c60 tx=0x7fdf08009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.239+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdefc055180 con 0x7fdf0c0b3100 2023-07-06T20:14:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.442+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf0c06ae40 con 0x7fdef006ad10 2023-07-06T20:14:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.453+0000 7fdf027fc700 1 -- 172.21.15.27:0/2387910733 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281426 (secure 0 0 0) 0x7fdf08019070 con 0x7fdef006ad10 2023-07-06T20:14:04.458 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdef006ad10 msgr2=0x7fdef006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdef006ad10 0x7fdef006d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009c60 tx=0x7fdf08009210 comp rx=0 tx=0).stop 2023-07-06T20:14:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 msgr2=0x7fdf0c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 0x7fdf0c0b3570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdef80047b0 tx=0x7fdefc001230 comp rx=0 tx=0).stop 2023-07-06T20:14:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 shutdown_connections 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdef006ad10 0x7fdef006d1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0c0ca030 0x7fdf0c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdef8003f70 0x7fdf0c14a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 --2- 172.21.15.27:0/2387910733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0c0b3100 0x7fdf0c0b3570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 >> 172.21.15.27:0/2387910733 conn(0x7fdf0c06c150 msgr2=0x7fdf0c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 shutdown_connections 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.457+0000 7fdf141ff700 1 -- 172.21.15.27:0/2387910733 wait complete. 2023-07-06T20:14:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:04.477 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-07-06T20:14:04.343802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218396,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756324,"statfs":{"total":773094113280,"available":772870475776,"internally_reserved":0,"allocated":2846720,"data_stored":1746541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001405"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012989+0000","last_change":"2023-07-06T20:13:59.012989+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012989+0000","last_undegraded":"2023-07-06T20:13:59.012989+0000","last_fullsized":"2023-07-06T20:13:59.012989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005032+0000","last_change":"2023-07-06T20:13:59.005032+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005032+0000","last_undegraded":"2023-07-06T20:13:59.005032+0000","last_fullsized":"2023-07-06T20:13:59.005032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.016201+0000","last_change":"2023-07-06T20:13:59.016201+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.016201+0000","last_undegraded":"2023-07-06T20:13:59.016201+0000","last_fullsized":"2023-07-06T20:13:59.016201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010988+0000","last_change":"2023-07-06T20:13:59.010988+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010988+0000","last_undegraded":"2023-07-06T20:13:59.010988+0000","last_fullsized":"2023-07-06T20:13:59.010988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012228+0000","last_change":"2023-07-06T20:13:59.012228+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012228+0000","last_undegraded":"2023-07-06T20:13:59.012228+0000","last_fullsized":"2023-07-06T20:13:59.012228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012354+0000","last_change":"2023-07-06T20:13:59.012354+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012354+0000","last_undegraded":"2023-07-06T20:13:59.012354+0000","last_fullsized":"2023-07-06T20:13:59.012354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009809+0000","last_change":"2023-07-06T20:13:59.009809+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009809+0000","last_undegraded":"2023-07-06T20:13:59.009809+0000","last_fullsized":"2023-07-06T20:13:59.009809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005602+0000","last_change":"2023-07-06T20:13:59.005602+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005602+0000","last_undegraded":"2023-07-06T20:13:59.005602+0000","last_fullsized":"2023-07-06T20:13:59.005602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.014144+0000","last_change":"2023-07-06T20:13:59.014144+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.014144+0000","last_undegraded":"2023-07-06T20:13:59.014144+0000","last_fullsized":"2023-07-06T20:13:59.014144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003803+0000","last_change":"2023-07-06T20:13:59.003803+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003803+0000","last_undegraded":"2023-07-06T20:13:59.003803+0000","last_fullsized":"2023-07-06T20:13:59.003803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009809+0000","last_change":"2023-07-06T20:13:59.009809+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009809+0000","last_undegraded":"2023-07-06T20:13:59.009809+0000","last_fullsized":"2023-07-06T20:13:59.009809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012052+0000","last_change":"2023-07-06T20:13:59.012052+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012052+0000","last_undegraded":"2023-07-06T20:13:59.012052+0000","last_fullsized":"2023-07-06T20:13:59.012052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009892+0000","last_change":"2023-07-06T20:13:59.009892+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009892+0000","last_undegraded":"2023-07-06T20:13:59.009892+0000","last_fullsized":"2023-07-06T20:13:59.009892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003488+0000","last_change":"2023-07-06T20:13:59.003488+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003488+0000","last_undegraded":"2023-07-06T20:13:59.003488+0000","last_fullsized":"2023-07-06T20:13:59.003488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009119+0000","last_change":"2023-07-06T20:13:59.009119+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009119+0000","last_undegraded":"2023-07-06T20:13:59.009119+0000","last_fullsized":"2023-07-06T20:13:59.009119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007127+0000","last_change":"2023-07-06T20:13:59.007127+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007127+0000","last_undegraded":"2023-07-06T20:13:59.007127+0000","last_fullsized":"2023-07-06T20:13:59.007127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013464+0000","last_change":"2023-07-06T20:13:59.013464+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013464+0000","last_undegraded":"2023-07-06T20:13:59.013464+0000","last_fullsized":"2023-07-06T20:13:59.013464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010207+0000","last_change":"2023-07-06T20:13:59.010207+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010207+0000","last_undegraded":"2023-07-06T20:13:59.010207+0000","last_fullsized":"2023-07-06T20:13:59.010207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013937+0000","last_change":"2023-07-06T20:13:59.013937+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013937+0000","last_undegraded":"2023-07-06T20:13:59.013937+0000","last_fullsized":"2023-07-06T20:13:59.013937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.014349+0000","last_change":"2023-07-06T20:13:59.014349+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.014349+0000","last_undegraded":"2023-07-06T20:13:59.014349+0000","last_fullsized":"2023-07-06T20:13:59.014349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010712+0000","last_change":"2023-07-06T20:13:59.010712+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010712+0000","last_undegraded":"2023-07-06T20:13:59.010712+0000","last_fullsized":"2023-07-06T20:13:59.010712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012508+0000","last_change":"2023-07-06T20:13:59.012508+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012508+0000","last_undegraded":"2023-07-06T20:13:59.012508+0000","last_fullsized":"2023-07-06T20:13:59.012508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011408+0000","last_change":"2023-07-06T20:13:59.011408+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011408+0000","last_undegraded":"2023-07-06T20:13:59.011408+0000","last_fullsized":"2023-07-06T20:13:59.011408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010528+0000","last_change":"2023-07-06T20:13:59.010528+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010528+0000","last_undegraded":"2023-07-06T20:13:59.010528+0000","last_fullsized":"2023-07-06T20:13:59.010528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007132+0000","last_change":"2023-07-06T20:13:59.007132+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007132+0000","last_undegraded":"2023-07-06T20:13:59.007132+0000","last_fullsized":"2023-07-06T20:13:59.007132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005624+0000","last_change":"2023-07-06T20:13:59.005624+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005624+0000","last_undegraded":"2023-07-06T20:13:59.005624+0000","last_fullsized":"2023-07-06T20:13:59.005624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005884+0000","last_change":"2023-07-06T20:13:59.005884+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005884+0000","last_undegraded":"2023-07-06T20:13:59.005884+0000","last_fullsized":"2023-07-06T20:13:59.005884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029960+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:13:59.029960+0000","last_peered":"2023-07-06T20:13:59.029960+0000","last_clean":"2023-07-06T20:13:59.029960+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:13:59.029960+0000","last_undegraded":"2023-07-06T20:13:59.029960+0000","last_fullsized":"2023-07-06T20:13:59.029960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009323+0000","last_change":"2023-07-06T20:13:59.009323+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009323+0000","last_undegraded":"2023-07-06T20:13:59.009323+0000","last_fullsized":"2023-07-06T20:13:59.009323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.023044+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:13:59.023044+0000","last_peered":"2023-07-06T20:13:59.023044+0000","last_clean":"2023-07-06T20:13:59.023044+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:13:59.023044+0000","last_undegraded":"2023-07-06T20:13:59.023044+0000","last_fullsized":"2023-07-06T20:13:59.023044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:29:39.278939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012065+0000","last_change":"2023-07-06T20:13:59.012065+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012065+0000","last_undegraded":"2023-07-06T20:13:59.012065+0000","last_fullsized":"2023-07-06T20:13:59.012065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030585+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:13:59.030585+0000","last_peered":"2023-07-06T20:13:59.030585+0000","last_clean":"2023-07-06T20:13:59.030585+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:13:59.030585+0000","last_undegraded":"2023-07-06T20:13:59.030585+0000","last_fullsized":"2023-07-06T20:13:59.030585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008685+0000","last_change":"2023-07-06T20:13:59.008685+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008685+0000","last_undegraded":"2023-07-06T20:13:59.008685+0000","last_fullsized":"2023-07-06T20:13:59.008685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.033112+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:13:59.033112+0000","last_peered":"2023-07-06T20:13:59.033112+0000","last_clean":"2023-07-06T20:13:59.033112+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:13:59.033112+0000","last_undegraded":"2023-07-06T20:13:59.033112+0000","last_fullsized":"2023-07-06T20:13:59.033112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006011+0000","last_change":"2023-07-06T20:13:59.006011+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006011+0000","last_undegraded":"2023-07-06T20:13:59.006011+0000","last_fullsized":"2023-07-06T20:13:59.006011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026007+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:13:59.026007+0000","last_peered":"2023-07-06T20:13:59.026007+0000","last_clean":"2023-07-06T20:13:59.026007+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:13:59.026007+0000","last_undegraded":"2023-07-06T20:13:59.026007+0000","last_fullsized":"2023-07-06T20:13:59.026007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:29:39.278939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006262+0000","last_change":"2023-07-06T20:13:59.006262+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006262+0000","last_undegraded":"2023-07-06T20:13:59.006262+0000","last_fullsized":"2023-07-06T20:13:59.006262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029908+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:13:59.029908+0000","last_peered":"2023-07-06T20:13:59.029908+0000","last_clean":"2023-07-06T20:13:59.029908+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:13:59.029908+0000","last_undegraded":"2023-07-06T20:13:59.029908+0000","last_fullsized":"2023-07-06T20:13:59.029908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:23.720299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004872+0000","last_change":"2023-07-06T20:13:59.004872+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004872+0000","last_undegraded":"2023-07-06T20:13:59.004872+0000","last_fullsized":"2023-07-06T20:13:59.004872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.032578+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:13:59.032578+0000","last_peered":"2023-07-06T20:13:59.032578+0000","last_clean":"2023-07-06T20:13:59.032578+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:13:59.032578+0000","last_undegraded":"2023-07-06T20:13:59.032578+0000","last_fullsized":"2023-07-06T20:13:59.032578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:03.837499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009309+0000","last_change":"2023-07-06T20:13:59.009309+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009309+0000","last_undegraded":"2023-07-06T20:13:59.009309+0000","last_fullsized":"2023-07-06T20:13:59.009309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030249+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:13:59.030249+0000","last_peered":"2023-07-06T20:13:59.030249+0000","last_clean":"2023-07-06T20:13:59.030249+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:13:59.030249+0000","last_undegraded":"2023-07-06T20:13:59.030249+0000","last_fullsized":"2023-07-06T20:13:59.030249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:23.879545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009417+0000","last_change":"2023-07-06T20:13:59.009417+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009417+0000","last_undegraded":"2023-07-06T20:13:59.009417+0000","last_fullsized":"2023-07-06T20:13:59.009417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027841+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:13:59.027841+0000","last_peered":"2023-07-06T20:13:59.027841+0000","last_clean":"2023-07-06T20:13:59.027841+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:13:59.027841+0000","last_undegraded":"2023-07-06T20:13:59.027841+0000","last_fullsized":"2023-07-06T20:13:59.027841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:30:28.001888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005392+0000","last_change":"2023-07-06T20:13:59.005392+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005392+0000","last_undegraded":"2023-07-06T20:13:59.005392+0000","last_fullsized":"2023-07-06T20:13:59.005392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030039+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:13:59.030039+0000","last_peered":"2023-07-06T20:13:59.030039+0000","last_clean":"2023-07-06T20:13:59.030039+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:13:59.030039+0000","last_undegraded":"2023-07-06T20:13:59.030039+0000","last_fullsized":"2023-07-06T20:13:59.030039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:16.523465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013625+0000","last_change":"2023-07-06T20:13:59.013625+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013625+0000","last_undegraded":"2023-07-06T20:13:59.013625+0000","last_fullsized":"2023-07-06T20:13:59.013625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026302+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:13:59.026302+0000","last_peered":"2023-07-06T20:13:59.026302+0000","last_clean":"2023-07-06T20:13:59.026302+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:13:59.026302+0000","last_undegraded":"2023-07-06T20:13:59.026302+0000","last_fullsized":"2023-07-06T20:13:59.026302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:34.097570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008850+0000","last_change":"2023-07-06T20:13:59.008850+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008850+0000","last_undegraded":"2023-07-06T20:13:59.008850+0000","last_fullsized":"2023-07-06T20:13:59.008850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030010+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:13:59.030010+0000","last_peered":"2023-07-06T20:13:59.030010+0000","last_clean":"2023-07-06T20:13:59.030010+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:13:59.030010+0000","last_undegraded":"2023-07-06T20:13:59.030010+0000","last_fullsized":"2023-07-06T20:13:59.030010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003351+0000","last_change":"2023-07-06T20:13:59.003351+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003351+0000","last_undegraded":"2023-07-06T20:13:59.003351+0000","last_fullsized":"2023-07-06T20:13:59.003351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030444+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:13:59.030444+0000","last_peered":"2023-07-06T20:13:59.030444+0000","last_clean":"2023-07-06T20:13:59.030444+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:13:59.030444+0000","last_undegraded":"2023-07-06T20:13:59.030444+0000","last_fullsized":"2023-07-06T20:13:59.030444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:52.468703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011134+0000","last_change":"2023-07-06T20:13:59.011134+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011134+0000","last_undegraded":"2023-07-06T20:13:59.011134+0000","last_fullsized":"2023-07-06T20:13:59.011134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030447+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:13:59.030447+0000","last_peered":"2023-07-06T20:13:59.030447+0000","last_clean":"2023-07-06T20:13:59.030447+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:13:59.030447+0000","last_undegraded":"2023-07-06T20:13:59.030447+0000","last_fullsized":"2023-07-06T20:13:59.030447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:06.172609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009636+0000","last_change":"2023-07-06T20:13:59.009636+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009636+0000","last_undegraded":"2023-07-06T20:13:59.009636+0000","last_fullsized":"2023-07-06T20:13:59.009636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026025+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:13:59.026025+0000","last_peered":"2023-07-06T20:13:59.026025+0000","last_clean":"2023-07-06T20:13:59.026025+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:13:59.026025+0000","last_undegraded":"2023-07-06T20:13:59.026025+0000","last_fullsized":"2023-07-06T20:13:59.026025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:26.071851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013060+0000","last_change":"2023-07-06T20:13:59.013060+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013060+0000","last_undegraded":"2023-07-06T20:13:59.013060+0000","last_fullsized":"2023-07-06T20:13:59.013060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030696+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:13:59.030696+0000","last_peered":"2023-07-06T20:13:59.030696+0000","last_clean":"2023-07-06T20:13:59.030696+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:13:59.030696+0000","last_undegraded":"2023-07-06T20:13:59.030696+0000","last_fullsized":"2023-07-06T20:13:59.030696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:39:35.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013846+0000","last_change":"2023-07-06T20:13:59.013846+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013846+0000","last_undegraded":"2023-07-06T20:13:59.013846+0000","last_fullsized":"2023-07-06T20:13:59.013846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.028721+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:13:59.028721+0000","last_peered":"2023-07-06T20:13:59.028721+0000","last_clean":"2023-07-06T20:13:59.028721+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:13:59.028721+0000","last_undegraded":"2023-07-06T20:13:59.028721+0000","last_fullsized":"2023-07-06T20:13:59.028721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007944+0000","last_change":"2023-07-06T20:13:59.007944+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007944+0000","last_undegraded":"2023-07-06T20:13:59.007944+0000","last_fullsized":"2023-07-06T20:13:59.007944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029821+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:13:59.029821+0000","last_peered":"2023-07-06T20:13:59.029821+0000","last_clean":"2023-07-06T20:13:59.029821+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:13:59.029821+0000","last_undegraded":"2023-07-06T20:13:59.029821+0000","last_fullsized":"2023-07-06T20:13:59.029821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008561+0000","last_change":"2023-07-06T20:13:59.008561+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008561+0000","last_undegraded":"2023-07-06T20:13:59.008561+0000","last_fullsized":"2023-07-06T20:13:59.008561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027856+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:13:59.027856+0000","last_peered":"2023-07-06T20:13:59.027856+0000","last_clean":"2023-07-06T20:13:59.027856+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:13:59.027856+0000","last_undegraded":"2023-07-06T20:13:59.027856+0000","last_fullsized":"2023-07-06T20:13:59.027856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006602+0000","last_change":"2023-07-06T20:13:59.006602+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006602+0000","last_undegraded":"2023-07-06T20:13:59.006602+0000","last_fullsized":"2023-07-06T20:13:59.006602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.000910+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:13:59.000910+0000","last_peered":"2023-07-06T20:13:59.000910+0000","last_clean":"2023-07-06T20:13:59.000910+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:13:59.000910+0000","last_undegraded":"2023-07-06T20:13:59.000910+0000","last_fullsized":"2023-07-06T20:13:59.000910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:43:39.980007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026037+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:13:59.026037+0000","last_peered":"2023-07-06T20:13:59.026037+0000","last_clean":"2023-07-06T20:13:59.026037+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:13:59.026037+0000","last_undegraded":"2023-07-06T20:13:59.026037+0000","last_fullsized":"2023-07-06T20:13:59.026037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:39:35.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004160+0000","last_change":"2023-07-06T20:13:59.004160+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004160+0000","last_undegraded":"2023-07-06T20:13:59.004160+0000","last_fullsized":"2023-07-06T20:13:59.004160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.006364+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:13:59.006364+0000","last_peered":"2023-07-06T20:13:59.006364+0000","last_clean":"2023-07-06T20:13:59.006364+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:13:59.006364+0000","last_undegraded":"2023-07-06T20:13:59.006364+0000","last_fullsized":"2023-07-06T20:13:59.006364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:52.498865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029932+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:13:59.029932+0000","last_peered":"2023-07-06T20:13:59.029932+0000","last_clean":"2023-07-06T20:13:59.029932+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:13:59.029932+0000","last_undegraded":"2023-07-06T20:13:59.029932+0000","last_fullsized":"2023-07-06T20:13:59.029932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004879+0000","last_change":"2023-07-06T20:13:59.004879+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004879+0000","last_undegraded":"2023-07-06T20:13:59.004879+0000","last_fullsized":"2023-07-06T20:13:59.004879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.013905+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:13:59.013905+0000","last_peered":"2023-07-06T20:13:59.013905+0000","last_clean":"2023-07-06T20:13:59.013905+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:13:59.013905+0000","last_undegraded":"2023-07-06T20:13:59.013905+0000","last_fullsized":"2023-07-06T20:13:59.013905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:01.861781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.023078+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:13:59.023078+0000","last_peered":"2023-07-06T20:13:59.023078+0000","last_clean":"2023-07-06T20:13:59.023078+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:13:59.023078+0000","last_undegraded":"2023-07-06T20:13:59.023078+0000","last_fullsized":"2023-07-06T20:13:59.023078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:58.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005224+0000","last_change":"2023-07-06T20:13:59.005224+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005224+0000","last_undegraded":"2023-07-06T20:13:59.005224+0000","last_fullsized":"2023-07-06T20:13:59.005224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.006412+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:13:59.006412+0000","last_peered":"2023-07-06T20:13:59.006412+0000","last_clean":"2023-07-06T20:13:59.006412+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:13:59.006412+0000","last_undegraded":"2023-07-06T20:13:59.006412+0000","last_fullsized":"2023-07-06T20:13:59.006412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.011123+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:13:59.011123+0000","last_peered":"2023-07-06T20:13:59.011123+0000","last_clean":"2023-07-06T20:13:59.011123+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:13:59.011123+0000","last_undegraded":"2023-07-06T20:13:59.011123+0000","last_fullsized":"2023-07-06T20:13:59.011123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:37.526594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027809+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:13:59.027809+0000","last_peered":"2023-07-06T20:13:59.027809+0000","last_clean":"2023-07-06T20:13:59.027809+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:13:59.027809+0000","last_undegraded":"2023-07-06T20:13:59.027809+0000","last_fullsized":"2023-07-06T20:13:59.027809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007609+0000","last_change":"2023-07-06T20:13:59.007609+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007609+0000","last_undegraded":"2023-07-06T20:13:59.007609+0000","last_fullsized":"2023-07-06T20:13:59.007609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.011294+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:13:59.011294+0000","last_peered":"2023-07-06T20:13:59.011294+0000","last_clean":"2023-07-06T20:13:59.011294+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:13:59.011294+0000","last_undegraded":"2023-07-06T20:13:59.011294+0000","last_fullsized":"2023-07-06T20:13:59.011294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:19.487180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030764+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:13:59.030764+0000","last_peered":"2023-07-06T20:13:59.030764+0000","last_clean":"2023-07-06T20:13:59.030764+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:13:59.030764+0000","last_undegraded":"2023-07-06T20:13:59.030764+0000","last_fullsized":"2023-07-06T20:13:59.030764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:49:35.134928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011373+0000","last_change":"2023-07-06T20:13:59.011373+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011373+0000","last_undegraded":"2023-07-06T20:13:59.011373+0000","last_fullsized":"2023-07-06T20:13:59.011373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.012089+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:13:59.012089+0000","last_peered":"2023-07-06T20:13:59.012089+0000","last_clean":"2023-07-06T20:13:59.012089+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:13:59.012089+0000","last_undegraded":"2023-07-06T20:13:59.012089+0000","last_fullsized":"2023-07-06T20:13:59.012089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:13.622948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.031040+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:13:59.031040+0000","last_peered":"2023-07-06T20:13:59.031040+0000","last_clean":"2023-07-06T20:13:59.031040+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:13:59.031040+0000","last_undegraded":"2023-07-06T20:13:59.031040+0000","last_fullsized":"2023-07-06T20:13:59.031040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:51.027369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010581+0000","last_change":"2023-07-06T20:13:59.010581+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010581+0000","last_undegraded":"2023-07-06T20:13:59.010581+0000","last_fullsized":"2023-07-06T20:13:59.010581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030732+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:13:59.030732+0000","last_peered":"2023-07-06T20:13:59.030732+0000","last_clean":"2023-07-06T20:13:59.030732+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:13:59.030732+0000","last_undegraded":"2023-07-06T20:13:59.030732+0000","last_fullsized":"2023-07-06T20:13:59.030732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:21.788254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.000844+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:13:59.000844+0000","last_peered":"2023-07-06T20:13:59.000844+0000","last_clean":"2023-07-06T20:13:59.000844+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:13:59.000844+0000","last_undegraded":"2023-07-06T20:13:59.000844+0000","last_fullsized":"2023-07-06T20:13:59.000844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:20.034130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006674+0000","last_change":"2023-07-06T20:13:59.006674+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006674+0000","last_undegraded":"2023-07-06T20:13:59.006674+0000","last_fullsized":"2023-07-06T20:13:59.006674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.008974+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:13:59.008974+0000","last_peered":"2023-07-06T20:13:59.008974+0000","last_clean":"2023-07-06T20:13:59.008974+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:13:59.008974+0000","last_undegraded":"2023-07-06T20:13:59.008974+0000","last_fullsized":"2023-07-06T20:13:59.008974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:01.861781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030664+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:13:59.030664+0000","last_peered":"2023-07-06T20:13:59.030664+0000","last_clean":"2023-07-06T20:13:59.030664+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:13:59.030664+0000","last_undegraded":"2023-07-06T20:13:59.030664+0000","last_fullsized":"2023-07-06T20:13:59.030664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010434+0000","last_change":"2023-07-06T20:13:59.010434+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010434+0000","last_undegraded":"2023-07-06T20:13:59.010434+0000","last_fullsized":"2023-07-06T20:13:59.010434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030809+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:13:59.030809+0000","last_peered":"2023-07-06T20:13:59.030809+0000","last_clean":"2023-07-06T20:13:59.030809+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:13:59.030809+0000","last_undegraded":"2023-07-06T20:13:59.030809+0000","last_fullsized":"2023-07-06T20:13:59.030809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009787+0000","last_change":"2023-07-06T20:13:59.009787+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009787+0000","last_undegraded":"2023-07-06T20:13:59.009787+0000","last_fullsized":"2023-07-06T20:13:59.009787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026598+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:13:59.026598+0000","last_peered":"2023-07-06T20:13:59.026598+0000","last_clean":"2023-07-06T20:13:59.026598+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:13:59.026598+0000","last_undegraded":"2023-07-06T20:13:59.026598+0000","last_fullsized":"2023-07-06T20:13:59.026598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:43:46.017729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007430+0000","last_change":"2023-07-06T20:13:59.007430+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007430+0000","last_undegraded":"2023-07-06T20:13:59.007430+0000","last_fullsized":"2023-07-06T20:13:59.007430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026056+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:13:59.026056+0000","last_peered":"2023-07-06T20:13:59.026056+0000","last_clean":"2023-07-06T20:13:59.026056+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:13:59.026056+0000","last_undegraded":"2023-07-06T20:13:59.026056+0000","last_fullsized":"2023-07-06T20:13:59.026056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:21.788254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007731+0000","last_change":"2023-07-06T20:13:59.007731+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007731+0000","last_undegraded":"2023-07-06T20:13:59.007731+0000","last_fullsized":"2023-07-06T20:13:59.007731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030711+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:13:59.030711+0000","last_peered":"2023-07-06T20:13:59.030711+0000","last_clean":"2023-07-06T20:13:59.030711+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:13:59.030711+0000","last_undegraded":"2023-07-06T20:13:59.030711+0000","last_fullsized":"2023-07-06T20:13:59.030711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:03.837499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009002+0000","last_change":"2023-07-06T20:13:59.009002+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009002+0000","last_undegraded":"2023-07-06T20:13:59.009002+0000","last_fullsized":"2023-07-06T20:13:59.009002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.033094+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:13:59.033094+0000","last_peered":"2023-07-06T20:13:59.033094+0000","last_clean":"2023-07-06T20:13:59.033094+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:13:59.033094+0000","last_undegraded":"2023-07-06T20:13:59.033094+0000","last_fullsized":"2023-07-06T20:13:59.033094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:34.097570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009585+0000","last_change":"2023-07-06T20:13:59.009585+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009585+0000","last_undegraded":"2023-07-06T20:13:59.009585+0000","last_fullsized":"2023-07-06T20:13:59.009585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008133+0000","last_change":"2023-07-06T20:13:59.008133+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008133+0000","last_undegraded":"2023-07-06T20:13:59.008133+0000","last_fullsized":"2023-07-06T20:13:59.008133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010894+0000","last_change":"2023-07-06T20:13:59.010894+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010894+0000","last_undegraded":"2023-07-06T20:13:59.010894+0000","last_fullsized":"2023-07-06T20:13:59.010894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011586+0000","last_change":"2023-07-06T20:13:59.011586+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011586+0000","last_undegraded":"2023-07-06T20:13:59.011586+0000","last_fullsized":"2023-07-06T20:13:59.011586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010976+0000","last_change":"2023-07-06T20:13:59.010976+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010976+0000","last_undegraded":"2023-07-06T20:13:59.010976+0000","last_fullsized":"2023-07-06T20:13:59.010976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005887+0000","last_change":"2023-07-06T20:13:59.005887+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005887+0000","last_undegraded":"2023-07-06T20:13:59.005887+0000","last_fullsized":"2023-07-06T20:13:59.005887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":666186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":658547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:04.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:04.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.719+0000 7ff69f7ff700 1 Processor -- start 2023-07-06T20:14:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.720+0000 7ff69f7ff700 1 -- start start 2023-07-06T20:14:04.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69f7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff6980b5c90 0x7ff6980d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69f7ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6980cf780 con 0x7ff69815fca0 2023-07-06T20:14:04.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69f7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6980cd490 con 0x7ff6980b6110 2023-07-06T20:14:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69f7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6980cd770 con 0x7ff6980b5c90 2023-07-06T20:14:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69cd9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff6980b5c90 0x7ff6980d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69cd9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff6980b5c90 0x7ff6980d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59226/0 (socket says 172.21.15.27:59226) 2023-07-06T20:14:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69cd9a700 1 -- 172.21.15.27:0/1343889639 learned_addr learned my addr 172.21.15.27:0/1343889639 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69cd9a700 1 -- 172.21.15.27:0/1343889639 >> v1:172.21.15.153:6789/0 conn(0x7ff6980b6110 legacy=0x7ff6980cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69cd9a700 1 -- 172.21.15.27:0/1343889639 >> v1:172.21.15.27:6789/0 conn(0x7ff69815fca0 legacy=0x7ff6980d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.721+0000 7ff69cd9a700 1 -- 172.21.15.27:0/1343889639 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6980cda50 con 0x7ff6980b5c90 2023-07-06T20:14:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.722+0000 7ff69cd9a700 1 --2- 172.21.15.27:0/1343889639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6980b5c90 0x7ff6980d2d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6880098f0 tx=0x7ff688009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53cd19922d93c08 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.722+0000 7ff68ffff700 1 -- 172.21.15.27:0/1343889639 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688004230 con 0x7ff6980b5c90 2023-07-06T20:14:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.722+0000 7ff68ffff700 1 -- 172.21.15.27:0/1343889639 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688004390 con 0x7ff6980b5c90 2023-07-06T20:14:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.722+0000 7ff68ffff700 1 -- 172.21.15.27:0/1343889639 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6880036a0 con 0x7ff6980b5c90 2023-07-06T20:14:04.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.722+0000 7ff69f7ff700 1 -- 172.21.15.27:0/1343889639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6980b5c90 msgr2=0x7ff6980d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.722+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/1343889639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6980b5c90 0x7ff6980d2d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6880098f0 tx=0x7ff688009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:04.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.723+0000 7ff69f7ff700 1 -- 172.21.15.27:0/1343889639 shutdown_connections 2023-07-06T20:14:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.723+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/1343889639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6980b5c90 0x7ff6980d2d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.723+0000 7ff69f7ff700 1 -- 172.21.15.27:0/1343889639 >> 172.21.15.27:0/1343889639 conn(0x7ff69806c150 msgr2=0x7ff69806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.723+0000 7ff69f7ff700 1 -- 172.21.15.27:0/1343889639 shutdown_connections 2023-07-06T20:14:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.723+0000 7ff69f7ff700 1 -- 172.21.15.27:0/1343889639 wait complete. 2023-07-06T20:14:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.724+0000 7ff69f7ff700 1 Processor -- start 2023-07-06T20:14:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.725+0000 7ff69f7ff700 1 -- start start 2023-07-06T20:14:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.725+0000 7ff69f7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6980b5c90 0x7ff698156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69f7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6980b6110 0x7ff6980b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69f7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 0x7ff6980cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69f7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980cb980 con 0x7ff69815fca0 2023-07-06T20:14:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69f7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980ca3a0 con 0x7ff6980b6110 2023-07-06T20:14:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69f7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980ca680 con 0x7ff6980b5c90 2023-07-06T20:14:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 0x7ff6980cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 0x7ff6980cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59232/0 (socket says 172.21.15.27:59232) 2023-07-06T20:14:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 -- 172.21.15.27:0/820176923 learned_addr learned my addr 172.21.15.27:0/820176923 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69d59b700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6980b5c90 0x7ff698156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69cd9a700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6980b6110 0x7ff6980b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 -- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6980b5c90 msgr2=0x7ff698156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6980b5c90 0x7ff698156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 -- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6980b6110 msgr2=0x7ff6980b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6980b6110 0x7ff6980b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 -- 172.21.15.27:0/820176923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688009550 con 0x7ff69815fca0 2023-07-06T20:14:04.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.726+0000 7ff69dd9c700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 0x7ff6980cb440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cc10 tx=0x7ff69000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:04.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.727+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69000abb0 con 0x7ff69815fca0 2023-07-06T20:14:04.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.727+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69000ad10 con 0x7ff69815fca0 2023-07-06T20:14:04.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.727+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690012660 con 0x7ff69815fca0 2023-07-06T20:14:04.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.727+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6980ca960 con 0x7ff69815fca0 2023-07-06T20:14:04.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.727+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6980caf20 con 0x7ff69815fca0 2023-07-06T20:14:04.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.728+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6980008d0 con 0x7ff69815fca0 2023-07-06T20:14:04.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.729+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6900127c0 con 0x7ff69815fca0 2023-07-06T20:14:04.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.730+0000 7ff68dffb700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff67c06ace0 0x7ff67c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:04.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.730+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff690089ca0 con 0x7ff69815fca0 2023-07-06T20:14:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.730+0000 7ff69d59b700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff67c06ace0 0x7ff67c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.731+0000 7ff69d59b700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff67c06ace0 0x7ff67c06d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6940016c0 tx=0x7ff694009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.734+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff690054cb0 con 0x7ff69815fca0 2023-07-06T20:14:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.935+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff69806ae40 con 0x7ff67c06ace0 2023-07-06T20:14:04.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.944+0000 7ff68dffb700 1 -- 172.21.15.27:0/820176923 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281426 (secure 0 0 0) 0x7ff6940050d0 con 0x7ff67c06ace0 2023-07-06T20:14:04.948 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:04.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff67c06ace0 msgr2=0x7ff67c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff67c06ace0 0x7ff67c06d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6940016c0 tx=0x7ff694009040 comp rx=0 tx=0).stop 2023-07-06T20:14:04.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 msgr2=0x7ff6980cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:04.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 0x7ff6980cb440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cc10 tx=0x7ff69000a060 comp rx=0 tx=0).stop 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 shutdown_connections 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff67c06ace0 0x7ff67c06d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff69815fca0 0x7ff6980cb440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6980b5c90 0x7ff698156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 --2- 172.21.15.27:0/820176923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6980b6110 0x7ff6980b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 >> 172.21.15.27:0/820176923 conn(0x7ff69806c150 msgr2=0x7ff69815a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:04.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 shutdown_connections 2023-07-06T20:14:04.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:04.948+0000 7ff69f7ff700 1 -- 172.21.15.27:0/820176923 wait complete. 2023-07-06T20:14:04.952 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:04.967 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-07-06T20:14:04.343802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218396,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756324,"statfs":{"total":773094113280,"available":772870475776,"internally_reserved":0,"allocated":2846720,"data_stored":1746541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001405"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012989+0000","last_change":"2023-07-06T20:13:59.012989+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012989+0000","last_undegraded":"2023-07-06T20:13:59.012989+0000","last_fullsized":"2023-07-06T20:13:59.012989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005032+0000","last_change":"2023-07-06T20:13:59.005032+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005032+0000","last_undegraded":"2023-07-06T20:13:59.005032+0000","last_fullsized":"2023-07-06T20:13:59.005032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.016201+0000","last_change":"2023-07-06T20:13:59.016201+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.016201+0000","last_undegraded":"2023-07-06T20:13:59.016201+0000","last_fullsized":"2023-07-06T20:13:59.016201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010988+0000","last_change":"2023-07-06T20:13:59.010988+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010988+0000","last_undegraded":"2023-07-06T20:13:59.010988+0000","last_fullsized":"2023-07-06T20:13:59.010988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012228+0000","last_change":"2023-07-06T20:13:59.012228+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012228+0000","last_undegraded":"2023-07-06T20:13:59.012228+0000","last_fullsized":"2023-07-06T20:13:59.012228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012354+0000","last_change":"2023-07-06T20:13:59.012354+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012354+0000","last_undegraded":"2023-07-06T20:13:59.012354+0000","last_fullsized":"2023-07-06T20:13:59.012354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009809+0000","last_change":"2023-07-06T20:13:59.009809+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009809+0000","last_undegraded":"2023-07-06T20:13:59.009809+0000","last_fullsized":"2023-07-06T20:13:59.009809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005602+0000","last_change":"2023-07-06T20:13:59.005602+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005602+0000","last_undegraded":"2023-07-06T20:13:59.005602+0000","last_fullsized":"2023-07-06T20:13:59.005602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.014144+0000","last_change":"2023-07-06T20:13:59.014144+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.014144+0000","last_undegraded":"2023-07-06T20:13:59.014144+0000","last_fullsized":"2023-07-06T20:13:59.014144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003803+0000","last_change":"2023-07-06T20:13:59.003803+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003803+0000","last_undegraded":"2023-07-06T20:13:59.003803+0000","last_fullsized":"2023-07-06T20:13:59.003803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009809+0000","last_change":"2023-07-06T20:13:59.009809+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009809+0000","last_undegraded":"2023-07-06T20:13:59.009809+0000","last_fullsized":"2023-07-06T20:13:59.009809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012052+0000","last_change":"2023-07-06T20:13:59.012052+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012052+0000","last_undegraded":"2023-07-06T20:13:59.012052+0000","last_fullsized":"2023-07-06T20:13:59.012052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009892+0000","last_change":"2023-07-06T20:13:59.009892+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009892+0000","last_undegraded":"2023-07-06T20:13:59.009892+0000","last_fullsized":"2023-07-06T20:13:59.009892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003488+0000","last_change":"2023-07-06T20:13:59.003488+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003488+0000","last_undegraded":"2023-07-06T20:13:59.003488+0000","last_fullsized":"2023-07-06T20:13:59.003488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009119+0000","last_change":"2023-07-06T20:13:59.009119+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009119+0000","last_undegraded":"2023-07-06T20:13:59.009119+0000","last_fullsized":"2023-07-06T20:13:59.009119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007127+0000","last_change":"2023-07-06T20:13:59.007127+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007127+0000","last_undegraded":"2023-07-06T20:13:59.007127+0000","last_fullsized":"2023-07-06T20:13:59.007127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013464+0000","last_change":"2023-07-06T20:13:59.013464+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013464+0000","last_undegraded":"2023-07-06T20:13:59.013464+0000","last_fullsized":"2023-07-06T20:13:59.013464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010207+0000","last_change":"2023-07-06T20:13:59.010207+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010207+0000","last_undegraded":"2023-07-06T20:13:59.010207+0000","last_fullsized":"2023-07-06T20:13:59.010207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013937+0000","last_change":"2023-07-06T20:13:59.013937+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013937+0000","last_undegraded":"2023-07-06T20:13:59.013937+0000","last_fullsized":"2023-07-06T20:13:59.013937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.014349+0000","last_change":"2023-07-06T20:13:59.014349+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.014349+0000","last_undegraded":"2023-07-06T20:13:59.014349+0000","last_fullsized":"2023-07-06T20:13:59.014349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010712+0000","last_change":"2023-07-06T20:13:59.010712+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010712+0000","last_undegraded":"2023-07-06T20:13:59.010712+0000","last_fullsized":"2023-07-06T20:13:59.010712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012508+0000","last_change":"2023-07-06T20:13:59.012508+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012508+0000","last_undegraded":"2023-07-06T20:13:59.012508+0000","last_fullsized":"2023-07-06T20:13:59.012508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011408+0000","last_change":"2023-07-06T20:13:59.011408+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011408+0000","last_undegraded":"2023-07-06T20:13:59.011408+0000","last_fullsized":"2023-07-06T20:13:59.011408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010528+0000","last_change":"2023-07-06T20:13:59.010528+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010528+0000","last_undegraded":"2023-07-06T20:13:59.010528+0000","last_fullsized":"2023-07-06T20:13:59.010528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007132+0000","last_change":"2023-07-06T20:13:59.007132+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007132+0000","last_undegraded":"2023-07-06T20:13:59.007132+0000","last_fullsized":"2023-07-06T20:13:59.007132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005624+0000","last_change":"2023-07-06T20:13:59.005624+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005624+0000","last_undegraded":"2023-07-06T20:13:59.005624+0000","last_fullsized":"2023-07-06T20:13:59.005624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005884+0000","last_change":"2023-07-06T20:13:59.005884+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005884+0000","last_undegraded":"2023-07-06T20:13:59.005884+0000","last_fullsized":"2023-07-06T20:13:59.005884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029960+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:13:59.029960+0000","last_peered":"2023-07-06T20:13:59.029960+0000","last_clean":"2023-07-06T20:13:59.029960+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:13:59.029960+0000","last_undegraded":"2023-07-06T20:13:59.029960+0000","last_fullsized":"2023-07-06T20:13:59.029960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009323+0000","last_change":"2023-07-06T20:13:59.009323+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009323+0000","last_undegraded":"2023-07-06T20:13:59.009323+0000","last_fullsized":"2023-07-06T20:13:59.009323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.023044+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:13:59.023044+0000","last_peered":"2023-07-06T20:13:59.023044+0000","last_clean":"2023-07-06T20:13:59.023044+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:13:59.023044+0000","last_undegraded":"2023-07-06T20:13:59.023044+0000","last_fullsized":"2023-07-06T20:13:59.023044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:29:39.278939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012065+0000","last_change":"2023-07-06T20:13:59.012065+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012065+0000","last_undegraded":"2023-07-06T20:13:59.012065+0000","last_fullsized":"2023-07-06T20:13:59.012065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030585+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:13:59.030585+0000","last_peered":"2023-07-06T20:13:59.030585+0000","last_clean":"2023-07-06T20:13:59.030585+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:13:59.030585+0000","last_undegraded":"2023-07-06T20:13:59.030585+0000","last_fullsized":"2023-07-06T20:13:59.030585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008685+0000","last_change":"2023-07-06T20:13:59.008685+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008685+0000","last_undegraded":"2023-07-06T20:13:59.008685+0000","last_fullsized":"2023-07-06T20:13:59.008685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.033112+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:13:59.033112+0000","last_peered":"2023-07-06T20:13:59.033112+0000","last_clean":"2023-07-06T20:13:59.033112+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:13:59.033112+0000","last_undegraded":"2023-07-06T20:13:59.033112+0000","last_fullsized":"2023-07-06T20:13:59.033112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006011+0000","last_change":"2023-07-06T20:13:59.006011+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006011+0000","last_undegraded":"2023-07-06T20:13:59.006011+0000","last_fullsized":"2023-07-06T20:13:59.006011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026007+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:13:59.026007+0000","last_peered":"2023-07-06T20:13:59.026007+0000","last_clean":"2023-07-06T20:13:59.026007+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:13:59.026007+0000","last_undegraded":"2023-07-06T20:13:59.026007+0000","last_fullsized":"2023-07-06T20:13:59.026007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:29:39.278939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006262+0000","last_change":"2023-07-06T20:13:59.006262+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006262+0000","last_undegraded":"2023-07-06T20:13:59.006262+0000","last_fullsized":"2023-07-06T20:13:59.006262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029908+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:13:59.029908+0000","last_peered":"2023-07-06T20:13:59.029908+0000","last_clean":"2023-07-06T20:13:59.029908+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:13:59.029908+0000","last_undegraded":"2023-07-06T20:13:59.029908+0000","last_fullsized":"2023-07-06T20:13:59.029908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:23.720299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004872+0000","last_change":"2023-07-06T20:13:59.004872+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004872+0000","last_undegraded":"2023-07-06T20:13:59.004872+0000","last_fullsized":"2023-07-06T20:13:59.004872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.032578+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:13:59.032578+0000","last_peered":"2023-07-06T20:13:59.032578+0000","last_clean":"2023-07-06T20:13:59.032578+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:13:59.032578+0000","last_undegraded":"2023-07-06T20:13:59.032578+0000","last_fullsized":"2023-07-06T20:13:59.032578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:03.837499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009309+0000","last_change":"2023-07-06T20:13:59.009309+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009309+0000","last_undegraded":"2023-07-06T20:13:59.009309+0000","last_fullsized":"2023-07-06T20:13:59.009309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030249+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:13:59.030249+0000","last_peered":"2023-07-06T20:13:59.030249+0000","last_clean":"2023-07-06T20:13:59.030249+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:13:59.030249+0000","last_undegraded":"2023-07-06T20:13:59.030249+0000","last_fullsized":"2023-07-06T20:13:59.030249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:23.879545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009417+0000","last_change":"2023-07-06T20:13:59.009417+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009417+0000","last_undegraded":"2023-07-06T20:13:59.009417+0000","last_fullsized":"2023-07-06T20:13:59.009417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027841+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:13:59.027841+0000","last_peered":"2023-07-06T20:13:59.027841+0000","last_clean":"2023-07-06T20:13:59.027841+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:13:59.027841+0000","last_undegraded":"2023-07-06T20:13:59.027841+0000","last_fullsized":"2023-07-06T20:13:59.027841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:30:28.001888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005392+0000","last_change":"2023-07-06T20:13:59.005392+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005392+0000","last_undegraded":"2023-07-06T20:13:59.005392+0000","last_fullsized":"2023-07-06T20:13:59.005392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030039+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:13:59.030039+0000","last_peered":"2023-07-06T20:13:59.030039+0000","last_clean":"2023-07-06T20:13:59.030039+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:13:59.030039+0000","last_undegraded":"2023-07-06T20:13:59.030039+0000","last_fullsized":"2023-07-06T20:13:59.030039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:16.523465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013625+0000","last_change":"2023-07-06T20:13:59.013625+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013625+0000","last_undegraded":"2023-07-06T20:13:59.013625+0000","last_fullsized":"2023-07-06T20:13:59.013625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026302+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:13:59.026302+0000","last_peered":"2023-07-06T20:13:59.026302+0000","last_clean":"2023-07-06T20:13:59.026302+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:13:59.026302+0000","last_undegraded":"2023-07-06T20:13:59.026302+0000","last_fullsized":"2023-07-06T20:13:59.026302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:34.097570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008850+0000","last_change":"2023-07-06T20:13:59.008850+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008850+0000","last_undegraded":"2023-07-06T20:13:59.008850+0000","last_fullsized":"2023-07-06T20:13:59.008850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030010+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:13:59.030010+0000","last_peered":"2023-07-06T20:13:59.030010+0000","last_clean":"2023-07-06T20:13:59.030010+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:13:59.030010+0000","last_undegraded":"2023-07-06T20:13:59.030010+0000","last_fullsized":"2023-07-06T20:13:59.030010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003351+0000","last_change":"2023-07-06T20:13:59.003351+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003351+0000","last_undegraded":"2023-07-06T20:13:59.003351+0000","last_fullsized":"2023-07-06T20:13:59.003351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030444+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:13:59.030444+0000","last_peered":"2023-07-06T20:13:59.030444+0000","last_clean":"2023-07-06T20:13:59.030444+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:13:59.030444+0000","last_undegraded":"2023-07-06T20:13:59.030444+0000","last_fullsized":"2023-07-06T20:13:59.030444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:52.468703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011134+0000","last_change":"2023-07-06T20:13:59.011134+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011134+0000","last_undegraded":"2023-07-06T20:13:59.011134+0000","last_fullsized":"2023-07-06T20:13:59.011134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030447+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:13:59.030447+0000","last_peered":"2023-07-06T20:13:59.030447+0000","last_clean":"2023-07-06T20:13:59.030447+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:13:59.030447+0000","last_undegraded":"2023-07-06T20:13:59.030447+0000","last_fullsized":"2023-07-06T20:13:59.030447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:06.172609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009636+0000","last_change":"2023-07-06T20:13:59.009636+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009636+0000","last_undegraded":"2023-07-06T20:13:59.009636+0000","last_fullsized":"2023-07-06T20:13:59.009636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026025+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:13:59.026025+0000","last_peered":"2023-07-06T20:13:59.026025+0000","last_clean":"2023-07-06T20:13:59.026025+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:13:59.026025+0000","last_undegraded":"2023-07-06T20:13:59.026025+0000","last_fullsized":"2023-07-06T20:13:59.026025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:26.071851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013060+0000","last_change":"2023-07-06T20:13:59.013060+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013060+0000","last_undegraded":"2023-07-06T20:13:59.013060+0000","last_fullsized":"2023-07-06T20:13:59.013060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030696+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:13:59.030696+0000","last_peered":"2023-07-06T20:13:59.030696+0000","last_clean":"2023-07-06T20:13:59.030696+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:13:59.030696+0000","last_undegraded":"2023-07-06T20:13:59.030696+0000","last_fullsized":"2023-07-06T20:13:59.030696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:39:35.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013846+0000","last_change":"2023-07-06T20:13:59.013846+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013846+0000","last_undegraded":"2023-07-06T20:13:59.013846+0000","last_fullsized":"2023-07-06T20:13:59.013846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.028721+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:13:59.028721+0000","last_peered":"2023-07-06T20:13:59.028721+0000","last_clean":"2023-07-06T20:13:59.028721+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:13:59.028721+0000","last_undegraded":"2023-07-06T20:13:59.028721+0000","last_fullsized":"2023-07-06T20:13:59.028721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007944+0000","last_change":"2023-07-06T20:13:59.007944+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007944+0000","last_undegraded":"2023-07-06T20:13:59.007944+0000","last_fullsized":"2023-07-06T20:13:59.007944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029821+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:13:59.029821+0000","last_peered":"2023-07-06T20:13:59.029821+0000","last_clean":"2023-07-06T20:13:59.029821+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:13:59.029821+0000","last_undegraded":"2023-07-06T20:13:59.029821+0000","last_fullsized":"2023-07-06T20:13:59.029821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008561+0000","last_change":"2023-07-06T20:13:59.008561+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008561+0000","last_undegraded":"2023-07-06T20:13:59.008561+0000","last_fullsized":"2023-07-06T20:13:59.008561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027856+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:13:59.027856+0000","last_peered":"2023-07-06T20:13:59.027856+0000","last_clean":"2023-07-06T20:13:59.027856+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:13:59.027856+0000","last_undegraded":"2023-07-06T20:13:59.027856+0000","last_fullsized":"2023-07-06T20:13:59.027856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006602+0000","last_change":"2023-07-06T20:13:59.006602+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006602+0000","last_undegraded":"2023-07-06T20:13:59.006602+0000","last_fullsized":"2023-07-06T20:13:59.006602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.000910+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:13:59.000910+0000","last_peered":"2023-07-06T20:13:59.000910+0000","last_clean":"2023-07-06T20:13:59.000910+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:13:59.000910+0000","last_undegraded":"2023-07-06T20:13:59.000910+0000","last_fullsized":"2023-07-06T20:13:59.000910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:43:39.980007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026037+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:13:59.026037+0000","last_peered":"2023-07-06T20:13:59.026037+0000","last_clean":"2023-07-06T20:13:59.026037+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:13:59.026037+0000","last_undegraded":"2023-07-06T20:13:59.026037+0000","last_fullsized":"2023-07-06T20:13:59.026037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:39:35.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004160+0000","last_change":"2023-07-06T20:13:59.004160+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004160+0000","last_undegraded":"2023-07-06T20:13:59.004160+0000","last_fullsized":"2023-07-06T20:13:59.004160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.006364+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:13:59.006364+0000","last_peered":"2023-07-06T20:13:59.006364+0000","last_clean":"2023-07-06T20:13:59.006364+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:13:59.006364+0000","last_undegraded":"2023-07-06T20:13:59.006364+0000","last_fullsized":"2023-07-06T20:13:59.006364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:52.498865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029932+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:13:59.029932+0000","last_peered":"2023-07-06T20:13:59.029932+0000","last_clean":"2023-07-06T20:13:59.029932+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:13:59.029932+0000","last_undegraded":"2023-07-06T20:13:59.029932+0000","last_fullsized":"2023-07-06T20:13:59.029932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004879+0000","last_change":"2023-07-06T20:13:59.004879+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004879+0000","last_undegraded":"2023-07-06T20:13:59.004879+0000","last_fullsized":"2023-07-06T20:13:59.004879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.013905+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:13:59.013905+0000","last_peered":"2023-07-06T20:13:59.013905+0000","last_clean":"2023-07-06T20:13:59.013905+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:13:59.013905+0000","last_undegraded":"2023-07-06T20:13:59.013905+0000","last_fullsized":"2023-07-06T20:13:59.013905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:01.861781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.023078+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:13:59.023078+0000","last_peered":"2023-07-06T20:13:59.023078+0000","last_clean":"2023-07-06T20:13:59.023078+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:13:59.023078+0000","last_undegraded":"2023-07-06T20:13:59.023078+0000","last_fullsized":"2023-07-06T20:13:59.023078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:58.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005224+0000","last_change":"2023-07-06T20:13:59.005224+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005224+0000","last_undegraded":"2023-07-06T20:13:59.005224+0000","last_fullsized":"2023-07-06T20:13:59.005224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.006412+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:13:59.006412+0000","last_peered":"2023-07-06T20:13:59.006412+0000","last_clean":"2023-07-06T20:13:59.006412+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:13:59.006412+0000","last_undegraded":"2023-07-06T20:13:59.006412+0000","last_fullsized":"2023-07-06T20:13:59.006412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.011123+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:13:59.011123+0000","last_peered":"2023-07-06T20:13:59.011123+0000","last_clean":"2023-07-06T20:13:59.011123+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:13:59.011123+0000","last_undegraded":"2023-07-06T20:13:59.011123+0000","last_fullsized":"2023-07-06T20:13:59.011123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:37.526594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027809+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:13:59.027809+0000","last_peered":"2023-07-06T20:13:59.027809+0000","last_clean":"2023-07-06T20:13:59.027809+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:13:59.027809+0000","last_undegraded":"2023-07-06T20:13:59.027809+0000","last_fullsized":"2023-07-06T20:13:59.027809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007609+0000","last_change":"2023-07-06T20:13:59.007609+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007609+0000","last_undegraded":"2023-07-06T20:13:59.007609+0000","last_fullsized":"2023-07-06T20:13:59.007609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.011294+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:13:59.011294+0000","last_peered":"2023-07-06T20:13:59.011294+0000","last_clean":"2023-07-06T20:13:59.011294+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:13:59.011294+0000","last_undegraded":"2023-07-06T20:13:59.011294+0000","last_fullsized":"2023-07-06T20:13:59.011294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:19.487180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030764+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:13:59.030764+0000","last_peered":"2023-07-06T20:13:59.030764+0000","last_clean":"2023-07-06T20:13:59.030764+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:13:59.030764+0000","last_undegraded":"2023-07-06T20:13:59.030764+0000","last_fullsized":"2023-07-06T20:13:59.030764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:49:35.134928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011373+0000","last_change":"2023-07-06T20:13:59.011373+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011373+0000","last_undegraded":"2023-07-06T20:13:59.011373+0000","last_fullsized":"2023-07-06T20:13:59.011373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.012089+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:13:59.012089+0000","last_peered":"2023-07-06T20:13:59.012089+0000","last_clean":"2023-07-06T20:13:59.012089+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:13:59.012089+0000","last_undegraded":"2023-07-06T20:13:59.012089+0000","last_fullsized":"2023-07-06T20:13:59.012089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:13.622948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.031040+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:13:59.031040+0000","last_peered":"2023-07-06T20:13:59.031040+0000","last_clean":"2023-07-06T20:13:59.031040+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:13:59.031040+0000","last_undegraded":"2023-07-06T20:13:59.031040+0000","last_fullsized":"2023-07-06T20:13:59.031040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:51.027369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010581+0000","last_change":"2023-07-06T20:13:59.010581+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010581+0000","last_undegraded":"2023-07-06T20:13:59.010581+0000","last_fullsized":"2023-07-06T20:13:59.010581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030732+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:13:59.030732+0000","last_peered":"2023-07-06T20:13:59.030732+0000","last_clean":"2023-07-06T20:13:59.030732+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:13:59.030732+0000","last_undegraded":"2023-07-06T20:13:59.030732+0000","last_fullsized":"2023-07-06T20:13:59.030732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:21.788254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.000844+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:13:59.000844+0000","last_peered":"2023-07-06T20:13:59.000844+0000","last_clean":"2023-07-06T20:13:59.000844+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:13:59.000844+0000","last_undegraded":"2023-07-06T20:13:59.000844+0000","last_fullsized":"2023-07-06T20:13:59.000844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:20.034130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006674+0000","last_change":"2023-07-06T20:13:59.006674+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006674+0000","last_undegraded":"2023-07-06T20:13:59.006674+0000","last_fullsized":"2023-07-06T20:13:59.006674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.008974+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:13:59.008974+0000","last_peered":"2023-07-06T20:13:59.008974+0000","last_clean":"2023-07-06T20:13:59.008974+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:13:59.008974+0000","last_undegraded":"2023-07-06T20:13:59.008974+0000","last_fullsized":"2023-07-06T20:13:59.008974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:01.861781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030664+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:13:59.030664+0000","last_peered":"2023-07-06T20:13:59.030664+0000","last_clean":"2023-07-06T20:13:59.030664+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:13:59.030664+0000","last_undegraded":"2023-07-06T20:13:59.030664+0000","last_fullsized":"2023-07-06T20:13:59.030664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010434+0000","last_change":"2023-07-06T20:13:59.010434+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010434+0000","last_undegraded":"2023-07-06T20:13:59.010434+0000","last_fullsized":"2023-07-06T20:13:59.010434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030809+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:13:59.030809+0000","last_peered":"2023-07-06T20:13:59.030809+0000","last_clean":"2023-07-06T20:13:59.030809+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:13:59.030809+0000","last_undegraded":"2023-07-06T20:13:59.030809+0000","last_fullsized":"2023-07-06T20:13:59.030809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009787+0000","last_change":"2023-07-06T20:13:59.009787+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009787+0000","last_undegraded":"2023-07-06T20:13:59.009787+0000","last_fullsized":"2023-07-06T20:13:59.009787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026598+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:13:59.026598+0000","last_peered":"2023-07-06T20:13:59.026598+0000","last_clean":"2023-07-06T20:13:59.026598+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:13:59.026598+0000","last_undegraded":"2023-07-06T20:13:59.026598+0000","last_fullsized":"2023-07-06T20:13:59.026598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:43:46.017729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007430+0000","last_change":"2023-07-06T20:13:59.007430+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007430+0000","last_undegraded":"2023-07-06T20:13:59.007430+0000","last_fullsized":"2023-07-06T20:13:59.007430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026056+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:13:59.026056+0000","last_peered":"2023-07-06T20:13:59.026056+0000","last_clean":"2023-07-06T20:13:59.026056+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:13:59.026056+0000","last_undegraded":"2023-07-06T20:13:59.026056+0000","last_fullsized":"2023-07-06T20:13:59.026056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:21.788254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007731+0000","last_change":"2023-07-06T20:13:59.007731+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007731+0000","last_undegraded":"2023-07-06T20:13:59.007731+0000","last_fullsized":"2023-07-06T20:13:59.007731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030711+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:13:59.030711+0000","last_peered":"2023-07-06T20:13:59.030711+0000","last_clean":"2023-07-06T20:13:59.030711+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:13:59.030711+0000","last_undegraded":"2023-07-06T20:13:59.030711+0000","last_fullsized":"2023-07-06T20:13:59.030711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:03.837499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009002+0000","last_change":"2023-07-06T20:13:59.009002+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009002+0000","last_undegraded":"2023-07-06T20:13:59.009002+0000","last_fullsized":"2023-07-06T20:13:59.009002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.033094+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:13:59.033094+0000","last_peered":"2023-07-06T20:13:59.033094+0000","last_clean":"2023-07-06T20:13:59.033094+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:13:59.033094+0000","last_undegraded":"2023-07-06T20:13:59.033094+0000","last_fullsized":"2023-07-06T20:13:59.033094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:34.097570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009585+0000","last_change":"2023-07-06T20:13:59.009585+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009585+0000","last_undegraded":"2023-07-06T20:13:59.009585+0000","last_fullsized":"2023-07-06T20:13:59.009585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008133+0000","last_change":"2023-07-06T20:13:59.008133+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008133+0000","last_undegraded":"2023-07-06T20:13:59.008133+0000","last_fullsized":"2023-07-06T20:13:59.008133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010894+0000","last_change":"2023-07-06T20:13:59.010894+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010894+0000","last_undegraded":"2023-07-06T20:13:59.010894+0000","last_fullsized":"2023-07-06T20:13:59.010894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011586+0000","last_change":"2023-07-06T20:13:59.011586+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011586+0000","last_undegraded":"2023-07-06T20:13:59.011586+0000","last_fullsized":"2023-07-06T20:13:59.011586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010976+0000","last_change":"2023-07-06T20:13:59.010976+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010976+0000","last_undegraded":"2023-07-06T20:13:59.010976+0000","last_fullsized":"2023-07-06T20:13:59.010976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005887+0000","last_change":"2023-07-06T20:13:59.005887+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005887+0000","last_undegraded":"2023-07-06T20:13:59.005887+0000","last_fullsized":"2023-07-06T20:13:59.005887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":666186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":658547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:04.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:05.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.212+0000 7f8e1c1ff700 1 Processor -- start 2023-07-06T20:14:05.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.212+0000 7f8e1c1ff700 1 -- start start 2023-07-06T20:14:05.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e1c1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e1415fca0 0x7f8e140d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e1c1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e140b6110 0x7f8e140cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e1c1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140cf780 con 0x7f8e140b5c90 2023-07-06T20:14:05.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e1c1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e140cd490 con 0x7f8e1415fca0 2023-07-06T20:14:05.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e1c1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e140cd770 con 0x7f8e140b6110 2023-07-06T20:14:05.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e19f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e1415fca0 0x7f8e140d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e19f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e1415fca0 0x7f8e140d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59242/0 (socket says 172.21.15.27:59242) 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e19f9b700 1 -- 172.21.15.27:0/3042129547 learned_addr learned my addr 172.21.15.27:0/3042129547 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.213+0000 7f8e1a79c700 1 --2- 172.21.15.27:0/3042129547 >> v2:172.21.15.153:3300/0 conn(0x7f8e140b6110 0x7f8e140cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e19f9b700 1 -- 172.21.15.27:0/3042129547 >> v2:172.21.15.153:3300/0 conn(0x7f8e140b6110 msgr2=0x7f8e140cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e19f9b700 1 --2- 172.21.15.27:0/3042129547 >> v2:172.21.15.153:3300/0 conn(0x7f8e140b6110 0x7f8e140cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e19f9b700 1 -- 172.21.15.27:0/3042129547 >> v1:172.21.15.153:6789/0 conn(0x7f8e140b5c90 legacy=0x7f8e140d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e19f9b700 1 -- 172.21.15.27:0/3042129547 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140cda50 con 0x7f8e1415fca0 2023-07-06T20:14:05.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e19f9b700 1 --2- 172.21.15.27:0/3042129547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e1415fca0 0x7f8e140d1420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e140cd410 tx=0x7f8e10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf2ab980bf95f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:05.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e18f99700 1 -- 172.21.15.27:0/3042129547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011830 con 0x7f8e1415fca0 2023-07-06T20:14:05.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e18f99700 1 -- 172.21.15.27:0/3042129547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10011990 con 0x7f8e1415fca0 2023-07-06T20:14:05.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.214+0000 7f8e18f99700 1 -- 172.21.15.27:0/3042129547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011d20 con 0x7f8e1415fca0 2023-07-06T20:14:05.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/3042129547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e1415fca0 msgr2=0x7f8e140d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/3042129547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e1415fca0 0x7f8e140d1420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e140cd410 tx=0x7f8e10009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/3042129547 shutdown_connections 2023-07-06T20:14:05.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/3042129547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e1415fca0 0x7f8e140d1420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/3042129547 >> v2:172.21.15.153:3300/0 conn(0x7f8e140b6110 0x7f8e140cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/3042129547 >> 172.21.15.27:0/3042129547 conn(0x7f8e1406c150 msgr2=0x7f8e1406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:05.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/3042129547 shutdown_connections 2023-07-06T20:14:05.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.215+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/3042129547 wait complete. 2023-07-06T20:14:05.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.217+0000 7f8e1c1ff700 1 Processor -- start 2023-07-06T20:14:05.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.218+0000 7f8e1c1ff700 1 -- start start 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e1c1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 0x7f8e14148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e1c1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e140b6110 0x7f8e140c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e1c1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e1415fca0 0x7f8e140c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e1c1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e14148c50 con 0x7f8e140b5c90 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e1c1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1414b230 con 0x7f8e1415fca0 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e1c1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1414b510 con 0x7f8e140b6110 2023-07-06T20:14:05.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e19f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 0x7f8e14148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e19f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 0x7f8e14148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59246/0 (socket says 172.21.15.27:59246) 2023-07-06T20:14:05.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e19f9b700 1 -- 172.21.15.27:0/447335737 learned_addr learned my addr 172.21.15.27:0/447335737 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:05.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e19f9b700 1 -- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e140b6110 msgr2=0x7f8e140c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.219+0000 7f8e19f9b700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e140b6110 0x7f8e140c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e19f9b700 1 -- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e1415fca0 msgr2=0x7f8e140c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e19f9b700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e1415fca0 0x7f8e140c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e19f9b700 1 -- 172.21.15.27:0/447335737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009550 con 0x7f8e140b5c90 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e19f9b700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 0x7f8e14148710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009c10 tx=0x7f8e10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100036a0 con 0x7f8e140b5c90 2023-07-06T20:14:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1001e070 con 0x7f8e140b5c90 2023-07-06T20:14:05.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1414b7f0 con 0x7f8e140b5c90 2023-07-06T20:14:05.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10003a60 con 0x7f8e140b5c90 2023-07-06T20:14:05.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.220+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1414bdb0 con 0x7f8e140b5c90 2023-07-06T20:14:05.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.221+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e140008d0 con 0x7f8e140b5c90 2023-07-06T20:14:05.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.222+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e10003bc0 con 0x7f8e140b5c90 2023-07-06T20:14:05.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.223+0000 7f8e0a7fc700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8df806ace0 0x7f8df806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.223+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e1008a350 con 0x7f8e140b5c90 2023-07-06T20:14:05.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.224+0000 7f8e1979a700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8df806ace0 0x7f8df806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.224+0000 7f8e1979a700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8df806ace0 0x7f8df806d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e04000f60 tx=0x7f8e04007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:05.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.228+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8e10055490 con 0x7f8e140b5c90 2023-07-06T20:14:05.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.431+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e1406ae40 con 0x7f8df806ace0 2023-07-06T20:14:05.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.439+0000 7f8e0a7fc700 1 -- 172.21.15.27:0/447335737 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281426 (secure 0 0 0) 0x7f8e04005080 con 0x7f8df806ace0 2023-07-06T20:14:05.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:05.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8df806ace0 msgr2=0x7f8df806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8df806ace0 0x7f8df806d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e04000f60 tx=0x7f8e04007a40 comp rx=0 tx=0).stop 2023-07-06T20:14:05.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 msgr2=0x7f8e14148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 0x7f8e14148710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009c10 tx=0x7f8e10004750 comp rx=0 tx=0).stop 2023-07-06T20:14:05.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 shutdown_connections 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8df806ace0 0x7f8df806d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e1415fca0 0x7f8e140c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e140b5c90 0x7f8e14148710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 --2- 172.21.15.27:0/447335737 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e140b6110 0x7f8e140c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 >> 172.21.15.27:0/447335737 conn(0x7f8e1406c150 msgr2=0x7f8e140b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 shutdown_connections 2023-07-06T20:14:05.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.444+0000 7f8e1c1ff700 1 -- 172.21.15.27:0/447335737 wait complete. 2023-07-06T20:14:05.448 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:05.463 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-07-06T20:14:04.343802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218396,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756324,"statfs":{"total":773094113280,"available":772870475776,"internally_reserved":0,"allocated":2846720,"data_stored":1746541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001405"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012989+0000","last_change":"2023-07-06T20:13:59.012989+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012989+0000","last_undegraded":"2023-07-06T20:13:59.012989+0000","last_fullsized":"2023-07-06T20:13:59.012989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005032+0000","last_change":"2023-07-06T20:13:59.005032+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005032+0000","last_undegraded":"2023-07-06T20:13:59.005032+0000","last_fullsized":"2023-07-06T20:13:59.005032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.016201+0000","last_change":"2023-07-06T20:13:59.016201+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.016201+0000","last_undegraded":"2023-07-06T20:13:59.016201+0000","last_fullsized":"2023-07-06T20:13:59.016201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010988+0000","last_change":"2023-07-06T20:13:59.010988+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010988+0000","last_undegraded":"2023-07-06T20:13:59.010988+0000","last_fullsized":"2023-07-06T20:13:59.010988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012228+0000","last_change":"2023-07-06T20:13:59.012228+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012228+0000","last_undegraded":"2023-07-06T20:13:59.012228+0000","last_fullsized":"2023-07-06T20:13:59.012228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012354+0000","last_change":"2023-07-06T20:13:59.012354+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012354+0000","last_undegraded":"2023-07-06T20:13:59.012354+0000","last_fullsized":"2023-07-06T20:13:59.012354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009809+0000","last_change":"2023-07-06T20:13:59.009809+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009809+0000","last_undegraded":"2023-07-06T20:13:59.009809+0000","last_fullsized":"2023-07-06T20:13:59.009809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005602+0000","last_change":"2023-07-06T20:13:59.005602+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005602+0000","last_undegraded":"2023-07-06T20:13:59.005602+0000","last_fullsized":"2023-07-06T20:13:59.005602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.014144+0000","last_change":"2023-07-06T20:13:59.014144+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.014144+0000","last_undegraded":"2023-07-06T20:13:59.014144+0000","last_fullsized":"2023-07-06T20:13:59.014144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003803+0000","last_change":"2023-07-06T20:13:59.003803+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003803+0000","last_undegraded":"2023-07-06T20:13:59.003803+0000","last_fullsized":"2023-07-06T20:13:59.003803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009809+0000","last_change":"2023-07-06T20:13:59.009809+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009809+0000","last_undegraded":"2023-07-06T20:13:59.009809+0000","last_fullsized":"2023-07-06T20:13:59.009809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012052+0000","last_change":"2023-07-06T20:13:59.012052+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012052+0000","last_undegraded":"2023-07-06T20:13:59.012052+0000","last_fullsized":"2023-07-06T20:13:59.012052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009892+0000","last_change":"2023-07-06T20:13:59.009892+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009892+0000","last_undegraded":"2023-07-06T20:13:59.009892+0000","last_fullsized":"2023-07-06T20:13:59.009892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003488+0000","last_change":"2023-07-06T20:13:59.003488+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003488+0000","last_undegraded":"2023-07-06T20:13:59.003488+0000","last_fullsized":"2023-07-06T20:13:59.003488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009119+0000","last_change":"2023-07-06T20:13:59.009119+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009119+0000","last_undegraded":"2023-07-06T20:13:59.009119+0000","last_fullsized":"2023-07-06T20:13:59.009119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007127+0000","last_change":"2023-07-06T20:13:59.007127+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007127+0000","last_undegraded":"2023-07-06T20:13:59.007127+0000","last_fullsized":"2023-07-06T20:13:59.007127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013464+0000","last_change":"2023-07-06T20:13:59.013464+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013464+0000","last_undegraded":"2023-07-06T20:13:59.013464+0000","last_fullsized":"2023-07-06T20:13:59.013464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010207+0000","last_change":"2023-07-06T20:13:59.010207+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010207+0000","last_undegraded":"2023-07-06T20:13:59.010207+0000","last_fullsized":"2023-07-06T20:13:59.010207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013937+0000","last_change":"2023-07-06T20:13:59.013937+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013937+0000","last_undegraded":"2023-07-06T20:13:59.013937+0000","last_fullsized":"2023-07-06T20:13:59.013937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.014349+0000","last_change":"2023-07-06T20:13:59.014349+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.014349+0000","last_undegraded":"2023-07-06T20:13:59.014349+0000","last_fullsized":"2023-07-06T20:13:59.014349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010712+0000","last_change":"2023-07-06T20:13:59.010712+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010712+0000","last_undegraded":"2023-07-06T20:13:59.010712+0000","last_fullsized":"2023-07-06T20:13:59.010712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012508+0000","last_change":"2023-07-06T20:13:59.012508+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012508+0000","last_undegraded":"2023-07-06T20:13:59.012508+0000","last_fullsized":"2023-07-06T20:13:59.012508+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011408+0000","last_change":"2023-07-06T20:13:59.011408+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011408+0000","last_undegraded":"2023-07-06T20:13:59.011408+0000","last_fullsized":"2023-07-06T20:13:59.011408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010528+0000","last_change":"2023-07-06T20:13:59.010528+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010528+0000","last_undegraded":"2023-07-06T20:13:59.010528+0000","last_fullsized":"2023-07-06T20:13:59.010528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007132+0000","last_change":"2023-07-06T20:13:59.007132+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007132+0000","last_undegraded":"2023-07-06T20:13:59.007132+0000","last_fullsized":"2023-07-06T20:13:59.007132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005624+0000","last_change":"2023-07-06T20:13:59.005624+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005624+0000","last_undegraded":"2023-07-06T20:13:59.005624+0000","last_fullsized":"2023-07-06T20:13:59.005624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005884+0000","last_change":"2023-07-06T20:13:59.005884+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005884+0000","last_undegraded":"2023-07-06T20:13:59.005884+0000","last_fullsized":"2023-07-06T20:13:59.005884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029960+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:13:59.029960+0000","last_peered":"2023-07-06T20:13:59.029960+0000","last_clean":"2023-07-06T20:13:59.029960+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:13:59.029960+0000","last_undegraded":"2023-07-06T20:13:59.029960+0000","last_fullsized":"2023-07-06T20:13:59.029960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009323+0000","last_change":"2023-07-06T20:13:59.009323+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009323+0000","last_undegraded":"2023-07-06T20:13:59.009323+0000","last_fullsized":"2023-07-06T20:13:59.009323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.023044+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:13:59.023044+0000","last_peered":"2023-07-06T20:13:59.023044+0000","last_clean":"2023-07-06T20:13:59.023044+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:13:59.023044+0000","last_undegraded":"2023-07-06T20:13:59.023044+0000","last_fullsized":"2023-07-06T20:13:59.023044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:29:39.278939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.012065+0000","last_change":"2023-07-06T20:13:59.012065+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.012065+0000","last_undegraded":"2023-07-06T20:13:59.012065+0000","last_fullsized":"2023-07-06T20:13:59.012065+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030585+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:13:59.030585+0000","last_peered":"2023-07-06T20:13:59.030585+0000","last_clean":"2023-07-06T20:13:59.030585+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:13:59.030585+0000","last_undegraded":"2023-07-06T20:13:59.030585+0000","last_fullsized":"2023-07-06T20:13:59.030585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008685+0000","last_change":"2023-07-06T20:13:59.008685+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008685+0000","last_undegraded":"2023-07-06T20:13:59.008685+0000","last_fullsized":"2023-07-06T20:13:59.008685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.033112+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:13:59.033112+0000","last_peered":"2023-07-06T20:13:59.033112+0000","last_clean":"2023-07-06T20:13:59.033112+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:13:59.033112+0000","last_undegraded":"2023-07-06T20:13:59.033112+0000","last_fullsized":"2023-07-06T20:13:59.033112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006011+0000","last_change":"2023-07-06T20:13:59.006011+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006011+0000","last_undegraded":"2023-07-06T20:13:59.006011+0000","last_fullsized":"2023-07-06T20:13:59.006011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026007+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:13:59.026007+0000","last_peered":"2023-07-06T20:13:59.026007+0000","last_clean":"2023-07-06T20:13:59.026007+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:13:59.026007+0000","last_undegraded":"2023-07-06T20:13:59.026007+0000","last_fullsized":"2023-07-06T20:13:59.026007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:29:39.278939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006262+0000","last_change":"2023-07-06T20:13:59.006262+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006262+0000","last_undegraded":"2023-07-06T20:13:59.006262+0000","last_fullsized":"2023-07-06T20:13:59.006262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029908+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:13:59.029908+0000","last_peered":"2023-07-06T20:13:59.029908+0000","last_clean":"2023-07-06T20:13:59.029908+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:13:59.029908+0000","last_undegraded":"2023-07-06T20:13:59.029908+0000","last_fullsized":"2023-07-06T20:13:59.029908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:23.720299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004872+0000","last_change":"2023-07-06T20:13:59.004872+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004872+0000","last_undegraded":"2023-07-06T20:13:59.004872+0000","last_fullsized":"2023-07-06T20:13:59.004872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.032578+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:13:59.032578+0000","last_peered":"2023-07-06T20:13:59.032578+0000","last_clean":"2023-07-06T20:13:59.032578+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:13:59.032578+0000","last_undegraded":"2023-07-06T20:13:59.032578+0000","last_fullsized":"2023-07-06T20:13:59.032578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:03.837499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009309+0000","last_change":"2023-07-06T20:13:59.009309+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009309+0000","last_undegraded":"2023-07-06T20:13:59.009309+0000","last_fullsized":"2023-07-06T20:13:59.009309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030249+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:13:59.030249+0000","last_peered":"2023-07-06T20:13:59.030249+0000","last_clean":"2023-07-06T20:13:59.030249+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:13:59.030249+0000","last_undegraded":"2023-07-06T20:13:59.030249+0000","last_fullsized":"2023-07-06T20:13:59.030249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:23.879545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009417+0000","last_change":"2023-07-06T20:13:59.009417+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009417+0000","last_undegraded":"2023-07-06T20:13:59.009417+0000","last_fullsized":"2023-07-06T20:13:59.009417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027841+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:13:59.027841+0000","last_peered":"2023-07-06T20:13:59.027841+0000","last_clean":"2023-07-06T20:13:59.027841+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:13:59.027841+0000","last_undegraded":"2023-07-06T20:13:59.027841+0000","last_fullsized":"2023-07-06T20:13:59.027841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:30:28.001888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005392+0000","last_change":"2023-07-06T20:13:59.005392+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005392+0000","last_undegraded":"2023-07-06T20:13:59.005392+0000","last_fullsized":"2023-07-06T20:13:59.005392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030039+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:13:59.030039+0000","last_peered":"2023-07-06T20:13:59.030039+0000","last_clean":"2023-07-06T20:13:59.030039+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:13:59.030039+0000","last_undegraded":"2023-07-06T20:13:59.030039+0000","last_fullsized":"2023-07-06T20:13:59.030039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:16.523465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013625+0000","last_change":"2023-07-06T20:13:59.013625+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013625+0000","last_undegraded":"2023-07-06T20:13:59.013625+0000","last_fullsized":"2023-07-06T20:13:59.013625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026302+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:13:59.026302+0000","last_peered":"2023-07-06T20:13:59.026302+0000","last_clean":"2023-07-06T20:13:59.026302+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:13:59.026302+0000","last_undegraded":"2023-07-06T20:13:59.026302+0000","last_fullsized":"2023-07-06T20:13:59.026302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:34.097570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008850+0000","last_change":"2023-07-06T20:13:59.008850+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008850+0000","last_undegraded":"2023-07-06T20:13:59.008850+0000","last_fullsized":"2023-07-06T20:13:59.008850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030010+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:13:59.030010+0000","last_peered":"2023-07-06T20:13:59.030010+0000","last_clean":"2023-07-06T20:13:59.030010+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:13:59.030010+0000","last_undegraded":"2023-07-06T20:13:59.030010+0000","last_fullsized":"2023-07-06T20:13:59.030010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.003351+0000","last_change":"2023-07-06T20:13:59.003351+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.003351+0000","last_undegraded":"2023-07-06T20:13:59.003351+0000","last_fullsized":"2023-07-06T20:13:59.003351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030444+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:13:59.030444+0000","last_peered":"2023-07-06T20:13:59.030444+0000","last_clean":"2023-07-06T20:13:59.030444+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:13:59.030444+0000","last_undegraded":"2023-07-06T20:13:59.030444+0000","last_fullsized":"2023-07-06T20:13:59.030444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:52.468703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011134+0000","last_change":"2023-07-06T20:13:59.011134+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011134+0000","last_undegraded":"2023-07-06T20:13:59.011134+0000","last_fullsized":"2023-07-06T20:13:59.011134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030447+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:13:59.030447+0000","last_peered":"2023-07-06T20:13:59.030447+0000","last_clean":"2023-07-06T20:13:59.030447+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:13:59.030447+0000","last_undegraded":"2023-07-06T20:13:59.030447+0000","last_fullsized":"2023-07-06T20:13:59.030447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:06.172609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009636+0000","last_change":"2023-07-06T20:13:59.009636+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009636+0000","last_undegraded":"2023-07-06T20:13:59.009636+0000","last_fullsized":"2023-07-06T20:13:59.009636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026025+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:13:59.026025+0000","last_peered":"2023-07-06T20:13:59.026025+0000","last_clean":"2023-07-06T20:13:59.026025+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:13:59.026025+0000","last_undegraded":"2023-07-06T20:13:59.026025+0000","last_fullsized":"2023-07-06T20:13:59.026025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:26.071851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013060+0000","last_change":"2023-07-06T20:13:59.013060+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013060+0000","last_undegraded":"2023-07-06T20:13:59.013060+0000","last_fullsized":"2023-07-06T20:13:59.013060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030696+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:13:59.030696+0000","last_peered":"2023-07-06T20:13:59.030696+0000","last_clean":"2023-07-06T20:13:59.030696+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:13:59.030696+0000","last_undegraded":"2023-07-06T20:13:59.030696+0000","last_fullsized":"2023-07-06T20:13:59.030696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:39:35.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.013846+0000","last_change":"2023-07-06T20:13:59.013846+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.013846+0000","last_undegraded":"2023-07-06T20:13:59.013846+0000","last_fullsized":"2023-07-06T20:13:59.013846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.028721+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:13:59.028721+0000","last_peered":"2023-07-06T20:13:59.028721+0000","last_clean":"2023-07-06T20:13:59.028721+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:13:59.028721+0000","last_undegraded":"2023-07-06T20:13:59.028721+0000","last_fullsized":"2023-07-06T20:13:59.028721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007944+0000","last_change":"2023-07-06T20:13:59.007944+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007944+0000","last_undegraded":"2023-07-06T20:13:59.007944+0000","last_fullsized":"2023-07-06T20:13:59.007944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029821+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:13:59.029821+0000","last_peered":"2023-07-06T20:13:59.029821+0000","last_clean":"2023-07-06T20:13:59.029821+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:13:59.029821+0000","last_undegraded":"2023-07-06T20:13:59.029821+0000","last_fullsized":"2023-07-06T20:13:59.029821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008561+0000","last_change":"2023-07-06T20:13:59.008561+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008561+0000","last_undegraded":"2023-07-06T20:13:59.008561+0000","last_fullsized":"2023-07-06T20:13:59.008561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027856+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:13:59.027856+0000","last_peered":"2023-07-06T20:13:59.027856+0000","last_clean":"2023-07-06T20:13:59.027856+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:13:59.027856+0000","last_undegraded":"2023-07-06T20:13:59.027856+0000","last_fullsized":"2023-07-06T20:13:59.027856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006602+0000","last_change":"2023-07-06T20:13:59.006602+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006602+0000","last_undegraded":"2023-07-06T20:13:59.006602+0000","last_fullsized":"2023-07-06T20:13:59.006602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.000910+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:13:59.000910+0000","last_peered":"2023-07-06T20:13:59.000910+0000","last_clean":"2023-07-06T20:13:59.000910+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:13:59.000910+0000","last_undegraded":"2023-07-06T20:13:59.000910+0000","last_fullsized":"2023-07-06T20:13:59.000910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:43:39.980007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026037+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:13:59.026037+0000","last_peered":"2023-07-06T20:13:59.026037+0000","last_clean":"2023-07-06T20:13:59.026037+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:13:59.026037+0000","last_undegraded":"2023-07-06T20:13:59.026037+0000","last_fullsized":"2023-07-06T20:13:59.026037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:39:35.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004160+0000","last_change":"2023-07-06T20:13:59.004160+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004160+0000","last_undegraded":"2023-07-06T20:13:59.004160+0000","last_fullsized":"2023-07-06T20:13:59.004160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.006364+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:13:59.006364+0000","last_peered":"2023-07-06T20:13:59.006364+0000","last_clean":"2023-07-06T20:13:59.006364+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:13:59.006364+0000","last_undegraded":"2023-07-06T20:13:59.006364+0000","last_fullsized":"2023-07-06T20:13:59.006364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:52.498865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.029932+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:13:59.029932+0000","last_peered":"2023-07-06T20:13:59.029932+0000","last_clean":"2023-07-06T20:13:59.029932+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:13:59.029932+0000","last_undegraded":"2023-07-06T20:13:59.029932+0000","last_fullsized":"2023-07-06T20:13:59.029932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.004879+0000","last_change":"2023-07-06T20:13:59.004879+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.004879+0000","last_undegraded":"2023-07-06T20:13:59.004879+0000","last_fullsized":"2023-07-06T20:13:59.004879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.013905+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:13:59.013905+0000","last_peered":"2023-07-06T20:13:59.013905+0000","last_clean":"2023-07-06T20:13:59.013905+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:13:59.013905+0000","last_undegraded":"2023-07-06T20:13:59.013905+0000","last_fullsized":"2023-07-06T20:13:59.013905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:01.861781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.023078+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:13:59.023078+0000","last_peered":"2023-07-06T20:13:59.023078+0000","last_clean":"2023-07-06T20:13:59.023078+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:13:59.023078+0000","last_undegraded":"2023-07-06T20:13:59.023078+0000","last_fullsized":"2023-07-06T20:13:59.023078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:58.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005224+0000","last_change":"2023-07-06T20:13:59.005224+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005224+0000","last_undegraded":"2023-07-06T20:13:59.005224+0000","last_fullsized":"2023-07-06T20:13:59.005224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.006412+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:13:59.006412+0000","last_peered":"2023-07-06T20:13:59.006412+0000","last_clean":"2023-07-06T20:13:59.006412+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:13:59.006412+0000","last_undegraded":"2023-07-06T20:13:59.006412+0000","last_fullsized":"2023-07-06T20:13:59.006412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.011123+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:13:59.011123+0000","last_peered":"2023-07-06T20:13:59.011123+0000","last_clean":"2023-07-06T20:13:59.011123+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:13:59.011123+0000","last_undegraded":"2023-07-06T20:13:59.011123+0000","last_fullsized":"2023-07-06T20:13:59.011123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:37.526594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.027809+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:13:59.027809+0000","last_peered":"2023-07-06T20:13:59.027809+0000","last_clean":"2023-07-06T20:13:59.027809+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:13:59.027809+0000","last_undegraded":"2023-07-06T20:13:59.027809+0000","last_fullsized":"2023-07-06T20:13:59.027809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:52.445607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007609+0000","last_change":"2023-07-06T20:13:59.007609+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007609+0000","last_undegraded":"2023-07-06T20:13:59.007609+0000","last_fullsized":"2023-07-06T20:13:59.007609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.011294+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:13:59.011294+0000","last_peered":"2023-07-06T20:13:59.011294+0000","last_clean":"2023-07-06T20:13:59.011294+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:13:59.011294+0000","last_undegraded":"2023-07-06T20:13:59.011294+0000","last_fullsized":"2023-07-06T20:13:59.011294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:19.487180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030764+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:13:59.030764+0000","last_peered":"2023-07-06T20:13:59.030764+0000","last_clean":"2023-07-06T20:13:59.030764+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:13:59.030764+0000","last_undegraded":"2023-07-06T20:13:59.030764+0000","last_fullsized":"2023-07-06T20:13:59.030764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:49:35.134928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011373+0000","last_change":"2023-07-06T20:13:59.011373+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011373+0000","last_undegraded":"2023-07-06T20:13:59.011373+0000","last_fullsized":"2023-07-06T20:13:59.011373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.012089+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:13:59.012089+0000","last_peered":"2023-07-06T20:13:59.012089+0000","last_clean":"2023-07-06T20:13:59.012089+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:13:59.012089+0000","last_undegraded":"2023-07-06T20:13:59.012089+0000","last_fullsized":"2023-07-06T20:13:59.012089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:13.622948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.031040+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:13:59.031040+0000","last_peered":"2023-07-06T20:13:59.031040+0000","last_clean":"2023-07-06T20:13:59.031040+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:13:59.031040+0000","last_undegraded":"2023-07-06T20:13:59.031040+0000","last_fullsized":"2023-07-06T20:13:59.031040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:51.027369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010581+0000","last_change":"2023-07-06T20:13:59.010581+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010581+0000","last_undegraded":"2023-07-06T20:13:59.010581+0000","last_fullsized":"2023-07-06T20:13:59.010581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030732+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:13:59.030732+0000","last_peered":"2023-07-06T20:13:59.030732+0000","last_clean":"2023-07-06T20:13:59.030732+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:13:59.030732+0000","last_undegraded":"2023-07-06T20:13:59.030732+0000","last_fullsized":"2023-07-06T20:13:59.030732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:21.788254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.000844+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:13:59.000844+0000","last_peered":"2023-07-06T20:13:59.000844+0000","last_clean":"2023-07-06T20:13:59.000844+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:13:59.000844+0000","last_undegraded":"2023-07-06T20:13:59.000844+0000","last_fullsized":"2023-07-06T20:13:59.000844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:20.034130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.006674+0000","last_change":"2023-07-06T20:13:59.006674+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.006674+0000","last_undegraded":"2023-07-06T20:13:59.006674+0000","last_fullsized":"2023-07-06T20:13:59.006674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.008974+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:13:59.008974+0000","last_peered":"2023-07-06T20:13:59.008974+0000","last_clean":"2023-07-06T20:13:59.008974+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:13:59.008974+0000","last_undegraded":"2023-07-06T20:13:59.008974+0000","last_fullsized":"2023-07-06T20:13:59.008974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:01.861781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030664+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:13:59.030664+0000","last_peered":"2023-07-06T20:13:59.030664+0000","last_clean":"2023-07-06T20:13:59.030664+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:13:59.030664+0000","last_undegraded":"2023-07-06T20:13:59.030664+0000","last_fullsized":"2023-07-06T20:13:59.030664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:43.144771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010434+0000","last_change":"2023-07-06T20:13:59.010434+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010434+0000","last_undegraded":"2023-07-06T20:13:59.010434+0000","last_fullsized":"2023-07-06T20:13:59.010434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030809+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:13:59.030809+0000","last_peered":"2023-07-06T20:13:59.030809+0000","last_clean":"2023-07-06T20:13:59.030809+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:13:59.030809+0000","last_undegraded":"2023-07-06T20:13:59.030809+0000","last_fullsized":"2023-07-06T20:13:59.030809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:04:07.899503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009787+0000","last_change":"2023-07-06T20:13:59.009787+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009787+0000","last_undegraded":"2023-07-06T20:13:59.009787+0000","last_fullsized":"2023-07-06T20:13:59.009787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026598+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:13:59.026598+0000","last_peered":"2023-07-06T20:13:59.026598+0000","last_clean":"2023-07-06T20:13:59.026598+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:13:59.026598+0000","last_undegraded":"2023-07-06T20:13:59.026598+0000","last_fullsized":"2023-07-06T20:13:59.026598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:43:46.017729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007430+0000","last_change":"2023-07-06T20:13:59.007430+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007430+0000","last_undegraded":"2023-07-06T20:13:59.007430+0000","last_fullsized":"2023-07-06T20:13:59.007430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.026056+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:13:59.026056+0000","last_peered":"2023-07-06T20:13:59.026056+0000","last_clean":"2023-07-06T20:13:59.026056+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:13:59.026056+0000","last_undegraded":"2023-07-06T20:13:59.026056+0000","last_fullsized":"2023-07-06T20:13:59.026056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:21.788254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.007731+0000","last_change":"2023-07-06T20:13:59.007731+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.007731+0000","last_undegraded":"2023-07-06T20:13:59.007731+0000","last_fullsized":"2023-07-06T20:13:59.007731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.030711+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:13:59.030711+0000","last_peered":"2023-07-06T20:13:59.030711+0000","last_clean":"2023-07-06T20:13:59.030711+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:13:59.030711+0000","last_undegraded":"2023-07-06T20:13:59.030711+0000","last_fullsized":"2023-07-06T20:13:59.030711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:03.837499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009002+0000","last_change":"2023-07-06T20:13:59.009002+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009002+0000","last_undegraded":"2023-07-06T20:13:59.009002+0000","last_fullsized":"2023-07-06T20:13:59.009002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T20:13:59.033094+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:13:59.033094+0000","last_peered":"2023-07-06T20:13:59.033094+0000","last_clean":"2023-07-06T20:13:59.033094+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:13:59.033094+0000","last_undegraded":"2023-07-06T20:13:59.033094+0000","last_fullsized":"2023-07-06T20:13:59.033094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:34.097570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.009585+0000","last_change":"2023-07-06T20:13:59.009585+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.009585+0000","last_undegraded":"2023-07-06T20:13:59.009585+0000","last_fullsized":"2023-07-06T20:13:59.009585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.008133+0000","last_change":"2023-07-06T20:13:59.008133+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.008133+0000","last_undegraded":"2023-07-06T20:13:59.008133+0000","last_fullsized":"2023-07-06T20:13:59.008133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010894+0000","last_change":"2023-07-06T20:13:59.010894+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010894+0000","last_undegraded":"2023-07-06T20:13:59.010894+0000","last_fullsized":"2023-07-06T20:13:59.010894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.011586+0000","last_change":"2023-07-06T20:13:59.011586+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.011586+0000","last_undegraded":"2023-07-06T20:13:59.011586+0000","last_fullsized":"2023-07-06T20:13:59.011586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.010976+0000","last_change":"2023-07-06T20:13:59.010976+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.010976+0000","last_undegraded":"2023-07-06T20:13:59.010976+0000","last_fullsized":"2023-07-06T20:13:59.010976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T20:13:59.005887+0000","last_change":"2023-07-06T20:13:59.005887+0000","last_active":"2023-07-06T20:13:58.995865+0000","last_peered":"2023-07-06T20:13:58.995865+0000","last_clean":"2023-07-06T20:13:58.995865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T20:13:59.005887+0000","last_undegraded":"2023-07-06T20:13:59.005887+0000","last_fullsized":"2023-07-06T20:13:59.005887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":666186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":658547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:05.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-07-06T20:14:05.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.012989+0000', 'last_change': '2023-07-06T20:13:59.012989+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.012989+0000', 'last_undegraded': '2023-07-06T20:13:59.012989+0000', 'last_fullsized': '2023-07-06T20:13:59.012989+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T20:14:05.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2023-07-06T20:14:05.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005032+0000', 'last_change': '2023-07-06T20:13:59.005032+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005032+0000', 'last_undegraded': '2023-07-06T20:13:59.005032+0000', 'last_fullsized': '2023-07-06T20:13:59.005032+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3d is not active+clean 2023-07-06T20:14:05.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.016201+0000', 'last_change': '2023-07-06T20:13:59.016201+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.016201+0000', 'last_undegraded': '2023-07-06T20:13:59.016201+0000', 'last_fullsized': '2023-07-06T20:13:59.016201+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3c is not active+clean 2023-07-06T20:14:05.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010988+0000', 'last_change': '2023-07-06T20:13:59.010988+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010988+0000', 'last_undegraded': '2023-07-06T20:13:59.010988+0000', 'last_fullsized': '2023-07-06T20:13:59.010988+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T20:14:05.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-07-06T20:14:05.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.012228+0000', 'last_change': '2023-07-06T20:13:59.012228+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.012228+0000', 'last_undegraded': '2023-07-06T20:13:59.012228+0000', 'last_fullsized': '2023-07-06T20:13:59.012228+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3a is not active+clean 2023-07-06T20:14:05.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.012354+0000', 'last_change': '2023-07-06T20:13:59.012354+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.012354+0000', 'last_undegraded': '2023-07-06T20:13:59.012354+0000', 'last_fullsized': '2023-07-06T20:13:59.012354+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.39 is not active+clean 2023-07-06T20:14:05.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009809+0000', 'last_change': '2023-07-06T20:13:59.009809+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009809+0000', 'last_undegraded': '2023-07-06T20:13:59.009809+0000', 'last_fullsized': '2023-07-06T20:13:59.009809+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T20:14:05.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.38 is not active+clean 2023-07-06T20:14:05.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005602+0000', 'last_change': '2023-07-06T20:13:59.005602+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005602+0000', 'last_undegraded': '2023-07-06T20:13:59.005602+0000', 'last_fullsized': '2023-07-06T20:13:59.005602+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-07-06T20:14:05.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.014144+0000', 'last_change': '2023-07-06T20:13:59.014144+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.014144+0000', 'last_undegraded': '2023-07-06T20:13:59.014144+0000', 'last_fullsized': '2023-07-06T20:13:59.014144+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.36 is not active+clean 2023-07-06T20:14:05.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.003803+0000', 'last_change': '2023-07-06T20:13:59.003803+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.003803+0000', 'last_undegraded': '2023-07-06T20:13:59.003803+0000', 'last_fullsized': '2023-07-06T20:13:59.003803+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.35 is not active+clean 2023-07-06T20:14:05.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009809+0000', 'last_change': '2023-07-06T20:13:59.009809+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009809+0000', 'last_undegraded': '2023-07-06T20:13:59.009809+0000', 'last_fullsized': '2023-07-06T20:13:59.009809+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-07-06T20:14:05.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.012052+0000', 'last_change': '2023-07-06T20:13:59.012052+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.012052+0000', 'last_undegraded': '2023-07-06T20:13:59.012052+0000', 'last_fullsized': '2023-07-06T20:13:59.012052+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T20:14:05.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-07-06T20:14:05.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009892+0000', 'last_change': '2023-07-06T20:13:59.009892+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009892+0000', 'last_undegraded': '2023-07-06T20:13:59.009892+0000', 'last_fullsized': '2023-07-06T20:13:59.009892+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T20:14:05.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.32 is not active+clean 2023-07-06T20:14:05.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.003488+0000', 'last_change': '2023-07-06T20:13:59.003488+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.003488+0000', 'last_undegraded': '2023-07-06T20:13:59.003488+0000', 'last_fullsized': '2023-07-06T20:13:59.003488+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.31 is not active+clean 2023-07-06T20:14:05.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009119+0000', 'last_change': '2023-07-06T20:13:59.009119+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009119+0000', 'last_undegraded': '2023-07-06T20:13:59.009119+0000', 'last_fullsized': '2023-07-06T20:13:59.009119+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T20:14:05.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.30 is not active+clean 2023-07-06T20:14:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.007127+0000', 'last_change': '2023-07-06T20:13:59.007127+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.007127+0000', 'last_undegraded': '2023-07-06T20:13:59.007127+0000', 'last_fullsized': '2023-07-06T20:13:59.007127+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-07-06T20:14:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.013464+0000', 'last_change': '2023-07-06T20:13:59.013464+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.013464+0000', 'last_undegraded': '2023-07-06T20:13:59.013464+0000', 'last_fullsized': '2023-07-06T20:13:59.013464+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2e is not active+clean 2023-07-06T20:14:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010207+0000', 'last_change': '2023-07-06T20:13:59.010207+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010207+0000', 'last_undegraded': '2023-07-06T20:13:59.010207+0000', 'last_fullsized': '2023-07-06T20:13:59.010207+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-07-06T20:14:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.013937+0000', 'last_change': '2023-07-06T20:13:59.013937+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.013937+0000', 'last_undegraded': '2023-07-06T20:13:59.013937+0000', 'last_fullsized': '2023-07-06T20:13:59.013937+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T20:14:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2c is not active+clean 2023-07-06T20:14:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.014349+0000', 'last_change': '2023-07-06T20:13:59.014349+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.014349+0000', 'last_undegraded': '2023-07-06T20:13:59.014349+0000', 'last_fullsized': '2023-07-06T20:13:59.014349+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2b is not active+clean 2023-07-06T20:14:05.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010712+0000', 'last_change': '2023-07-06T20:13:59.010712+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010712+0000', 'last_undegraded': '2023-07-06T20:13:59.010712+0000', 'last_fullsized': '2023-07-06T20:13:59.010712+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T20:14:05.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-07-06T20:14:05.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.012508+0000', 'last_change': '2023-07-06T20:13:59.012508+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.012508+0000', 'last_undegraded': '2023-07-06T20:13:59.012508+0000', 'last_fullsized': '2023-07-06T20:13:59.012508+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2023-07-06T20:14:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.011408+0000', 'last_change': '2023-07-06T20:13:59.011408+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.011408+0000', 'last_undegraded': '2023-07-06T20:13:59.011408+0000', 'last_fullsized': '2023-07-06T20:13:59.011408+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.28 is not active+clean 2023-07-06T20:14:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010528+0000', 'last_change': '2023-07-06T20:13:59.010528+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010528+0000', 'last_undegraded': '2023-07-06T20:13:59.010528+0000', 'last_fullsized': '2023-07-06T20:13:59.010528+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T20:14:05.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2023-07-06T20:14:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.007132+0000', 'last_change': '2023-07-06T20:13:59.007132+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.007132+0000', 'last_undegraded': '2023-07-06T20:13:59.007132+0000', 'last_fullsized': '2023-07-06T20:13:59.007132+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2023-07-06T20:14:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005624+0000', 'last_change': '2023-07-06T20:13:59.005624+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005624+0000', 'last_undegraded': '2023-07-06T20:13:59.005624+0000', 'last_fullsized': '2023-07-06T20:13:59.005624+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.25 is not active+clean 2023-07-06T20:14:05.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005884+0000', 'last_change': '2023-07-06T20:13:59.005884+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005884+0000', 'last_undegraded': '2023-07-06T20:13:59.005884+0000', 'last_fullsized': '2023-07-06T20:13:59.005884+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-07-06T20:14:05.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009323+0000', 'last_change': '2023-07-06T20:13:59.009323+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009323+0000', 'last_undegraded': '2023-07-06T20:13:59.009323+0000', 'last_fullsized': '2023-07-06T20:13:59.009323+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-07-06T20:14:05.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.012065+0000', 'last_change': '2023-07-06T20:13:59.012065+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.012065+0000', 'last_undegraded': '2023-07-06T20:13:59.012065+0000', 'last_fullsized': '2023-07-06T20:13:59.012065+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-07-06T20:14:05.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.008685+0000', 'last_change': '2023-07-06T20:13:59.008685+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.008685+0000', 'last_undegraded': '2023-07-06T20:13:59.008685+0000', 'last_fullsized': '2023-07-06T20:13:59.008685+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T20:14:05.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-07-06T20:14:05.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.006011+0000', 'last_change': '2023-07-06T20:13:59.006011+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.006011+0000', 'last_undegraded': '2023-07-06T20:13:59.006011+0000', 'last_fullsized': '2023-07-06T20:13:59.006011+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-07-06T20:14:05.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.006262+0000', 'last_change': '2023-07-06T20:13:59.006262+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.006262+0000', 'last_undegraded': '2023-07-06T20:13:59.006262+0000', 'last_fullsized': '2023-07-06T20:13:59.006262+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-07-06T20:14:05.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.004872+0000', 'last_change': '2023-07-06T20:13:59.004872+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.004872+0000', 'last_undegraded': '2023-07-06T20:13:59.004872+0000', 'last_fullsized': '2023-07-06T20:13:59.004872+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-07-06T20:14:05.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009309+0000', 'last_change': '2023-07-06T20:13:59.009309+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009309+0000', 'last_undegraded': '2023-07-06T20:13:59.009309+0000', 'last_fullsized': '2023-07-06T20:13:59.009309+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-07-06T20:14:05.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009417+0000', 'last_change': '2023-07-06T20:13:59.009417+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009417+0000', 'last_undegraded': '2023-07-06T20:13:59.009417+0000', 'last_fullsized': '2023-07-06T20:13:59.009417+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-07-06T20:14:05.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005392+0000', 'last_change': '2023-07-06T20:13:59.005392+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005392+0000', 'last_undegraded': '2023-07-06T20:13:59.005392+0000', 'last_fullsized': '2023-07-06T20:13:59.005392+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-07-06T20:14:05.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.013625+0000', 'last_change': '2023-07-06T20:13:59.013625+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.013625+0000', 'last_undegraded': '2023-07-06T20:13:59.013625+0000', 'last_fullsized': '2023-07-06T20:13:59.013625+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-07-06T20:14:05.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.008850+0000', 'last_change': '2023-07-06T20:13:59.008850+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.008850+0000', 'last_undegraded': '2023-07-06T20:13:59.008850+0000', 'last_fullsized': '2023-07-06T20:13:59.008850+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-07-06T20:14:05.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.003351+0000', 'last_change': '2023-07-06T20:13:59.003351+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.003351+0000', 'last_undegraded': '2023-07-06T20:13:59.003351+0000', 'last_fullsized': '2023-07-06T20:13:59.003351+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T20:14:05.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-07-06T20:14:05.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.011134+0000', 'last_change': '2023-07-06T20:13:59.011134+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.011134+0000', 'last_undegraded': '2023-07-06T20:13:59.011134+0000', 'last_fullsized': '2023-07-06T20:13:59.011134+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-07-06T20:14:05.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009636+0000', 'last_change': '2023-07-06T20:13:59.009636+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009636+0000', 'last_undegraded': '2023-07-06T20:13:59.009636+0000', 'last_fullsized': '2023-07-06T20:13:59.009636+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T20:14:05.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-07-06T20:14:05.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.013060+0000', 'last_change': '2023-07-06T20:13:59.013060+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.013060+0000', 'last_undegraded': '2023-07-06T20:13:59.013060+0000', 'last_fullsized': '2023-07-06T20:13:59.013060+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-07-06T20:14:05.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.013846+0000', 'last_change': '2023-07-06T20:13:59.013846+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.013846+0000', 'last_undegraded': '2023-07-06T20:13:59.013846+0000', 'last_fullsized': '2023-07-06T20:13:59.013846+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-07-06T20:14:05.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.007944+0000', 'last_change': '2023-07-06T20:13:59.007944+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.007944+0000', 'last_undegraded': '2023-07-06T20:13:59.007944+0000', 'last_fullsized': '2023-07-06T20:13:59.007944+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-07-06T20:14:05.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.008561+0000', 'last_change': '2023-07-06T20:13:59.008561+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.008561+0000', 'last_undegraded': '2023-07-06T20:13:59.008561+0000', 'last_fullsized': '2023-07-06T20:13:59.008561+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-07-06T20:14:05.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.006602+0000', 'last_change': '2023-07-06T20:13:59.006602+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.006602+0000', 'last_undegraded': '2023-07-06T20:13:59.006602+0000', 'last_fullsized': '2023-07-06T20:13:59.006602+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-07-06T20:14:05.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.004160+0000', 'last_change': '2023-07-06T20:13:59.004160+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.004160+0000', 'last_undegraded': '2023-07-06T20:13:59.004160+0000', 'last_fullsized': '2023-07-06T20:13:59.004160+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-07-06T20:14:05.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.004879+0000', 'last_change': '2023-07-06T20:13:59.004879+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.004879+0000', 'last_undegraded': '2023-07-06T20:13:59.004879+0000', 'last_fullsized': '2023-07-06T20:13:59.004879+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-07-06T20:14:05.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005224+0000', 'last_change': '2023-07-06T20:13:59.005224+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005224+0000', 'last_undegraded': '2023-07-06T20:13:59.005224+0000', 'last_fullsized': '2023-07-06T20:13:59.005224+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-07-06T20:14:05.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.007609+0000', 'last_change': '2023-07-06T20:13:59.007609+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.007609+0000', 'last_undegraded': '2023-07-06T20:13:59.007609+0000', 'last_fullsized': '2023-07-06T20:13:59.007609+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T20:14:05.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-07-06T20:14:05.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.011373+0000', 'last_change': '2023-07-06T20:13:59.011373+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.011373+0000', 'last_undegraded': '2023-07-06T20:13:59.011373+0000', 'last_fullsized': '2023-07-06T20:13:59.011373+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-07-06T20:14:05.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010581+0000', 'last_change': '2023-07-06T20:13:59.010581+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010581+0000', 'last_undegraded': '2023-07-06T20:13:59.010581+0000', 'last_fullsized': '2023-07-06T20:13:59.010581+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T20:14:05.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-07-06T20:14:05.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.006674+0000', 'last_change': '2023-07-06T20:13:59.006674+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.006674+0000', 'last_undegraded': '2023-07-06T20:13:59.006674+0000', 'last_fullsized': '2023-07-06T20:13:59.006674+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-07-06T20:14:05.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010434+0000', 'last_change': '2023-07-06T20:13:59.010434+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010434+0000', 'last_undegraded': '2023-07-06T20:13:59.010434+0000', 'last_fullsized': '2023-07-06T20:13:59.010434+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-07-06T20:14:05.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009787+0000', 'last_change': '2023-07-06T20:13:59.009787+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009787+0000', 'last_undegraded': '2023-07-06T20:13:59.009787+0000', 'last_fullsized': '2023-07-06T20:13:59.009787+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T20:14:05.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-07-06T20:14:05.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.007430+0000', 'last_change': '2023-07-06T20:13:59.007430+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.007430+0000', 'last_undegraded': '2023-07-06T20:13:59.007430+0000', 'last_fullsized': '2023-07-06T20:13:59.007430+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-07-06T20:14:05.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.007731+0000', 'last_change': '2023-07-06T20:13:59.007731+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.007731+0000', 'last_undegraded': '2023-07-06T20:13:59.007731+0000', 'last_fullsized': '2023-07-06T20:13:59.007731+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-07-06T20:14:05.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009002+0000', 'last_change': '2023-07-06T20:13:59.009002+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009002+0000', 'last_undegraded': '2023-07-06T20:13:59.009002+0000', 'last_fullsized': '2023-07-06T20:13:59.009002+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-07-06T20:14:05.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.009585+0000', 'last_change': '2023-07-06T20:13:59.009585+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.009585+0000', 'last_undegraded': '2023-07-06T20:13:59.009585+0000', 'last_fullsized': '2023-07-06T20:13:59.009585+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.20 is not active+clean 2023-07-06T20:14:05.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.008133+0000', 'last_change': '2023-07-06T20:13:59.008133+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.008133+0000', 'last_undegraded': '2023-07-06T20:13:59.008133+0000', 'last_fullsized': '2023-07-06T20:13:59.008133+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T20:14:05.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2023-07-06T20:14:05.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010894+0000', 'last_change': '2023-07-06T20:13:59.010894+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010894+0000', 'last_undegraded': '2023-07-06T20:13:59.010894+0000', 'last_fullsized': '2023-07-06T20:13:59.010894+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.22 is not active+clean 2023-07-06T20:14:05.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.011586+0000', 'last_change': '2023-07-06T20:13:59.011586+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.011586+0000', 'last_undegraded': '2023-07-06T20:13:59.011586+0000', 'last_fullsized': '2023-07-06T20:13:59.011586+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T20:14:05.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2023-07-06T20:14:05.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.010976+0000', 'last_change': '2023-07-06T20:13:59.010976+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.010976+0000', 'last_undegraded': '2023-07-06T20:13:59.010976+0000', 'last_fullsized': '2023-07-06T20:13:59.010976+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T20:14:05.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2023-07-06T20:14:05.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2023-07-06T20:13:59.005887+0000', 'last_change': '2023-07-06T20:13:59.005887+0000', 'last_active': '2023-07-06T20:13:58.995865+0000', 'last_peered': '2023-07-06T20:13:58.995865+0000', 'last_clean': '2023-07-06T20:13:58.995865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T20:13:59.005887+0000', 'last_undegraded': '2023-07-06T20:13:59.005887+0000', 'last_fullsized': '2023-07-06T20:13:59.005887+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'last_clean_scrub_stamp': '2023-07-06T20:13:58.995865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T20:14:05.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-06T20:14:05.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.707+0000 7fe3aedff700 1 Processor -- start 2023-07-06T20:14:05.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.707+0000 7fe3aedff700 1 -- start start 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3aedff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3aedff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe3a80b60e0 0x7fe3a80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3aedff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80cf780 con 0x7fe3a815fca0 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3aedff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a80cd490 con 0x7fe3a80b60e0 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3aedff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80cd770 con 0x7fe3a80b5c90 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3acb9b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe3a815fca0 0x7fe3a80d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46092/0 (socket says 172.21.15.27:46092) 2023-07-06T20:14:05.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3acb9b700 1 -- 172.21.15.27:0/3007813959 learned_addr learned my addr 172.21.15.27:0/3007813959 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3a7fff700 1 --2- 172.21.15.27:0/3007813959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.708+0000 7fe3ad39c700 1 --2- 172.21.15.27:0/3007813959 >> v2:172.21.15.153:3300/0 conn(0x7fe3a80b60e0 0x7fe3a80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a77fe700 1 -- 172.21.15.27:0/3007813959 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465324191 0 0) 0x7fe3a80cf780 con 0x7fe3a815fca0 2023-07-06T20:14:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a77fe700 1 -- 172.21.15.27:0/3007813959 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3980035b0 con 0x7fe3a815fca0 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a7fff700 1 -- 172.21.15.27:0/3007813959 >> v2:172.21.15.153:3300/0 conn(0x7fe3a80b60e0 msgr2=0x7fe3a80d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a7fff700 1 --2- 172.21.15.27:0/3007813959 >> v2:172.21.15.153:3300/0 conn(0x7fe3a80b60e0 0x7fe3a80d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a7fff700 1 -- 172.21.15.27:0/3007813959 >> v1:172.21.15.27:6789/0 conn(0x7fe3a815fca0 legacy=0x7fe3a80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a7fff700 1 -- 172.21.15.27:0/3007813959 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80cda50 con 0x7fe3a80b5c90 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a7fff700 1 --2- 172.21.15.27:0/3007813959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a80d2d40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe394009840 tx=0x7fe394009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abbfd4decf88ccf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.709+0000 7fe3a77fe700 1 -- 172.21.15.27:0/3007813959 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394011830 con 0x7fe3a80b5c90 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3a77fe700 1 -- 172.21.15.27:0/3007813959 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394011990 con 0x7fe3a80b5c90 2023-07-06T20:14:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3a77fe700 1 -- 172.21.15.27:0/3007813959 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394011d20 con 0x7fe3a80b5c90 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 -- 172.21.15.27:0/3007813959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 msgr2=0x7fe3a80d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 --2- 172.21.15.27:0/3007813959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a80d2d40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe394009840 tx=0x7fe394009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 -- 172.21.15.27:0/3007813959 shutdown_connections 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 --2- 172.21.15.27:0/3007813959 >> v2:172.21.15.153:3300/0 conn(0x7fe3a80b60e0 0x7fe3a80d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 --2- 172.21.15.27:0/3007813959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a80d2d40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 -- 172.21.15.27:0/3007813959 >> 172.21.15.27:0/3007813959 conn(0x7fe3a806c150 msgr2=0x7fe3a806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 -- 172.21.15.27:0/3007813959 shutdown_connections 2023-07-06T20:14:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.710+0000 7fe3aedff700 1 -- 172.21.15.27:0/3007813959 wait complete. 2023-07-06T20:14:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.712+0000 7fe3aedff700 1 Processor -- start 2023-07-06T20:14:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.712+0000 7fe3aedff700 1 -- start start 2023-07-06T20:14:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3aedff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a81f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3aedff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a80b60e0 0x7fe3a8157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3aedff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 0x7fe3a815d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3aedff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b28c0 con 0x7fe3a815fca0 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3aedff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b2b10 con 0x7fe3a80b60e0 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3aedff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b2df0 con 0x7fe3a80b5c90 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3ad39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 0x7fe3a815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3ad39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 0x7fe3a815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59262/0 (socket says 172.21.15.27:59262) 2023-07-06T20:14:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3ad39c700 1 -- 172.21.15.27:0/1277067854 learned_addr learned my addr 172.21.15.27:0/1277067854 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3a7fff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a80b60e0 0x7fe3a8157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.713+0000 7fe3acb9b700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a81f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3ad39c700 1 -- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 msgr2=0x7fe3a81f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3ad39c700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a81f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3ad39c700 1 -- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a80b60e0 msgr2=0x7fe3a8157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3ad39c700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a80b60e0 0x7fe3a8157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:05.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3ad39c700 1 -- 172.21.15.27:0/1277067854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3940094a0 con 0x7fe3a815fca0 2023-07-06T20:14:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3ad39c700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 0x7fe3a815d690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000efc0 tx=0x7fe3a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000cc90 con 0x7fe3a815fca0 2023-07-06T20:14:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a000f040 con 0x7fe3a815fca0 2023-07-06T20:14:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a80b30d0 con 0x7fe3a815fca0 2023-07-06T20:14:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.714+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00146c0 con 0x7fe3a815fca0 2023-07-06T20:14:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.715+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a80b3690 con 0x7fe3a815fca0 2023-07-06T20:14:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.716+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a80008d0 con 0x7fe3a815fca0 2023-07-06T20:14:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.716+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3a0014820 con 0x7fe3a815fca0 2023-07-06T20:14:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.718+0000 7fe3a57fa700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe38c06ace0 0x7fe38c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.718+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3a008a060 con 0x7fe3a815fca0 2023-07-06T20:14:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.718+0000 7fe3acb9b700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe38c06ace0 0x7fe38c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.719+0000 7fe3acb9b700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe38c06ace0 0x7fe38c06d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007fd0 tx=0x7fe39c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:05.722+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3a00bc310 con 0x7fe3a815fca0 2023-07-06T20:14:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.015+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe3a80cec70 con 0x7fe3a815fca0 2023-07-06T20:14:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.016+0000 7fe3a57fa700 1 -- 172.21.15.27:0/1277067854 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1032 (secure 0 0 0) 0x7fe3a0018420 con 0x7fe3a815fca0 2023-07-06T20:14:06.018 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:06.018 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":34,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1688674423,"num_in_osds":8,"osd_in_since":1688674414,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":223637504,"bytes_avail":772870475776,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4299}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-07-06T20:13:31.025735+0000","services":{}},"progress_events":{}} 2023-07-06T20:14:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe38c06ace0 msgr2=0x7fe38c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe38c06ace0 0x7fe38c06d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007fd0 tx=0x7fe39c009040 comp rx=0 tx=0).stop 2023-07-06T20:14:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 msgr2=0x7fe3a815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 0x7fe3a815d690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000efc0 tx=0x7fe3a000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 shutdown_connections 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe38c06ace0 0x7fe38c06d190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3a815fca0 0x7fe3a815d690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3a80b5c90 0x7fe3a81f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 --2- 172.21.15.27:0/1277067854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a80b60e0 0x7fe3a8157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 >> 172.21.15.27:0/1277067854 conn(0x7fe3a806c150 msgr2=0x7fe3a80b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:06.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 shutdown_connections 2023-07-06T20:14:06.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.018+0000 7fe3aedff700 1 -- 172.21.15.27:0/1277067854 wait complete. 2023-07-06T20:14:06.038 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-06T20:14:06.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:06.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.239+0000 7f260a1ff700 1 Processor -- start 2023-07-06T20:14:06.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.239+0000 7f260a1ff700 1 -- start start 2023-07-06T20:14:06.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.240+0000 7f260a1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f260415fca0 0x7f26040d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f260a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f26040d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f260a1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26040b60e0 0x7f26040d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f260a1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040cf780 con 0x7f260415fca0 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f260a1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040cd490 con 0x7f26040b60e0 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f260a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040cd770 con 0x7f26040b5c90 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f260415fca0 0x7f26040d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f260415fca0 0x7f26040d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59264/0 (socket says 172.21.15.27:59264) 2023-07-06T20:14:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 -- 172.21.15.27:0/84133581 learned_addr learned my addr 172.21.15.27:0/84133581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f2603fff700 1 --2- 172.21.15.27:0/84133581 >> v2:172.21.15.153:3300/0 conn(0x7f26040b60e0 0x7f26040d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f2602ffd700 1 --2- 172.21.15.27:0/84133581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f26040d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 -- 172.21.15.27:0/84133581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 msgr2=0x7f26040d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 --2- 172.21.15.27:0/84133581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f26040d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 -- 172.21.15.27:0/84133581 >> v2:172.21.15.153:3300/0 conn(0x7f26040b60e0 msgr2=0x7f26040d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 --2- 172.21.15.27:0/84133581 >> v2:172.21.15.153:3300/0 conn(0x7f26040b60e0 0x7f26040d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.241+0000 7f26037fe700 1 -- 172.21.15.27:0/84133581 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040cda50 con 0x7f260415fca0 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f26037fe700 1 --2- 172.21.15.27:0/84133581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f260415fca0 0x7f26040d1420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26040b8f80 tx=0x7f25f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70449bca0f8f21cb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f26027fc700 1 -- 172.21.15.27:0/84133581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011830 con 0x7f260415fca0 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f26027fc700 1 -- 172.21.15.27:0/84133581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4011990 con 0x7f260415fca0 2023-07-06T20:14:06.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f26027fc700 1 -- 172.21.15.27:0/84133581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011d20 con 0x7f260415fca0 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 -- 172.21.15.27:0/84133581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f260415fca0 msgr2=0x7f26040d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 --2- 172.21.15.27:0/84133581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f260415fca0 0x7f26040d1420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26040b8f80 tx=0x7f25f4009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 -- 172.21.15.27:0/84133581 shutdown_connections 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 --2- 172.21.15.27:0/84133581 >> v2:172.21.15.153:3300/0 conn(0x7f26040b60e0 0x7f26040d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 --2- 172.21.15.27:0/84133581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f260415fca0 0x7f26040d1420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 --2- 172.21.15.27:0/84133581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f26040d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.242+0000 7f260a1ff700 1 -- 172.21.15.27:0/84133581 >> 172.21.15.27:0/84133581 conn(0x7f260406c150 msgr2=0x7f260406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.243+0000 7f260a1ff700 1 -- 172.21.15.27:0/84133581 shutdown_connections 2023-07-06T20:14:06.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.243+0000 7f260a1ff700 1 -- 172.21.15.27:0/84133581 wait complete. 2023-07-06T20:14:06.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.244+0000 7f260a1ff700 1 Processor -- start 2023-07-06T20:14:06.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.244+0000 7f260a1ff700 1 -- start start 2023-07-06T20:14:06.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.245+0000 7f260a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f2604156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.245+0000 7f260a1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 0x7f26040cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.245+0000 7f260a1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f260415fca0 0x7f26040cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.245+0000 7f260a1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260415b730 con 0x7f26040b60e0 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.245+0000 7f260a1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604158b50 con 0x7f260415fca0 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.245+0000 7f260a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260415b980 con 0x7f26040b5c90 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 0x7f26040cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 0x7f26040cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59270/0 (socket says 172.21.15.27:59270) 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 -- 172.21.15.27:0/2292903256 learned_addr learned my addr 172.21.15.27:0/2292903256 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f26037fe700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f2604156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2603fff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f260415fca0 0x7f26040cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 -- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 msgr2=0x7f2604156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f2604156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 -- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f260415fca0 msgr2=0x7f26040cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f260415fca0 0x7f26040cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 -- 172.21.15.27:0/2292903256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f4009550 con 0x7f26040b60e0 2023-07-06T20:14:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f2602ffd700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 0x7f26040cb880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e930 tx=0x7f25f800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8013680 con 0x7f26040b60e0 2023-07-06T20:14:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.246+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f80137e0 con 0x7f26040b60e0 2023-07-06T20:14:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.247+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8013b70 con 0x7f26040b60e0 2023-07-06T20:14:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.247+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260415bc60 con 0x7f26040b60e0 2023-07-06T20:14:06.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.247+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041f1cf0 con 0x7f26040b60e0 2023-07-06T20:14:06.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.248+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260415c2b0 con 0x7f26040b60e0 2023-07-06T20:14:06.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.249+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25f8013cd0 con 0x7f26040b60e0 2023-07-06T20:14:06.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.250+0000 7f25ebfff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f25f006ace0 0x7f25f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:06.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.250+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25f8089400 con 0x7f26040b60e0 2023-07-06T20:14:06.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.250+0000 7f26037fe700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f25f006ace0 0x7f25f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:06.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.251+0000 7f26037fe700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f25f006ace0 0x7f25f006d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25f4014040 tx=0x7f25f4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:06.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.254+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f25f80544a0 con 0x7f26040b60e0 2023-07-06T20:14:06.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.459+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f260406ae40 con 0x7f25f006ace0 2023-07-06T20:14:06.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.468+0000 7f25ebfff700 1 -- 172.21.15.27:0/2292903256 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286739 (secure 0 0 0) 0x7f25f4004040 con 0x7f25f006ace0 2023-07-06T20:14:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f25f006ace0 msgr2=0x7f25f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f25f006ace0 0x7f25f006d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25f4014040 tx=0x7f25f4008030 comp rx=0 tx=0).stop 2023-07-06T20:14:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 msgr2=0x7f26040cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 0x7f26040cb880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e930 tx=0x7f25f800ec40 comp rx=0 tx=0).stop 2023-07-06T20:14:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 shutdown_connections 2023-07-06T20:14:06.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f25f006ace0 0x7f25f006d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f260415fca0 0x7f26040cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b5c90 0x7f2604156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 --2- 172.21.15.27:0/2292903256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26040b60e0 0x7f26040cb880 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 >> 172.21.15.27:0/2292903256 conn(0x7f260406c150 msgr2=0x7f26040ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 shutdown_connections 2023-07-06T20:14:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:06.472+0000 7f260a1ff700 1 -- 172.21.15.27:0/2292903256 wait complete. 2023-07-06T20:14:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:06.492 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-07-06T20:14:06.344214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.344763"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238965+0000","last_change":"2023-07-06T20:14:00.041890+0000","last_active":"2023-07-06T20:14:01.238965+0000","last_peered":"2023-07-06T20:14:01.238965+0000","last_clean":"2023-07-06T20:14:01.238965+0000","last_became_active":"2023-07-06T20:14:00.041518+0000","last_became_peered":"2023-07-06T20:14:00.041518+0000","last_unstale":"2023-07-06T20:14:01.238965+0000","last_undegraded":"2023-07-06T20:14:01.238965+0000","last_fullsized":"2023-07-06T20:14:01.238965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:26.527907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239844+0000","last_change":"2023-07-06T20:14:00.067412+0000","last_active":"2023-07-06T20:14:01.239844+0000","last_peered":"2023-07-06T20:14:01.239844+0000","last_clean":"2023-07-06T20:14:01.239844+0000","last_became_active":"2023-07-06T20:14:00.067104+0000","last_became_peered":"2023-07-06T20:14:00.067104+0000","last_unstale":"2023-07-06T20:14:01.239844+0000","last_undegraded":"2023-07-06T20:14:01.239844+0000","last_fullsized":"2023-07-06T20:14:01.239844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047383+0000","last_change":"2023-07-06T20:14:00.056588+0000","last_active":"2023-07-06T20:14:01.047383+0000","last_peered":"2023-07-06T20:14:01.047383+0000","last_clean":"2023-07-06T20:14:01.047383+0000","last_became_active":"2023-07-06T20:14:00.056267+0000","last_became_peered":"2023-07-06T20:14:00.056267+0000","last_unstale":"2023-07-06T20:14:01.047383+0000","last_undegraded":"2023-07-06T20:14:01.047383+0000","last_fullsized":"2023-07-06T20:14:01.047383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240208+0000","last_change":"2023-07-06T20:14:00.058980+0000","last_active":"2023-07-06T20:14:01.240208+0000","last_peered":"2023-07-06T20:14:01.240208+0000","last_clean":"2023-07-06T20:14:01.240208+0000","last_became_active":"2023-07-06T20:14:00.058707+0000","last_became_peered":"2023-07-06T20:14:00.058707+0000","last_unstale":"2023-07-06T20:14:01.240208+0000","last_undegraded":"2023-07-06T20:14:01.240208+0000","last_fullsized":"2023-07-06T20:14:01.240208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:14.285291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239684+0000","last_change":"2023-07-06T20:14:00.064519+0000","last_active":"2023-07-06T20:14:01.239684+0000","last_peered":"2023-07-06T20:14:01.239684+0000","last_clean":"2023-07-06T20:14:01.239684+0000","last_became_active":"2023-07-06T20:14:00.064199+0000","last_became_peered":"2023-07-06T20:14:00.064199+0000","last_unstale":"2023-07-06T20:14:01.239684+0000","last_undegraded":"2023-07-06T20:14:01.239684+0000","last_fullsized":"2023-07-06T20:14:01.239684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047036+0000","last_change":"2023-07-06T20:14:00.056564+0000","last_active":"2023-07-06T20:14:01.047036+0000","last_peered":"2023-07-06T20:14:01.047036+0000","last_clean":"2023-07-06T20:14:01.047036+0000","last_became_active":"2023-07-06T20:14:00.056292+0000","last_became_peered":"2023-07-06T20:14:00.056292+0000","last_unstale":"2023-07-06T20:14:01.047036+0000","last_undegraded":"2023-07-06T20:14:01.047036+0000","last_fullsized":"2023-07-06T20:14:01.047036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239166+0000","last_change":"2023-07-06T20:14:00.068005+0000","last_active":"2023-07-06T20:14:01.239166+0000","last_peered":"2023-07-06T20:14:01.239166+0000","last_clean":"2023-07-06T20:14:01.239166+0000","last_became_active":"2023-07-06T20:14:00.067567+0000","last_became_peered":"2023-07-06T20:14:00.067567+0000","last_unstale":"2023-07-06T20:14:01.239166+0000","last_undegraded":"2023-07-06T20:14:01.239166+0000","last_fullsized":"2023-07-06T20:14:01.239166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239597+0000","last_change":"2023-07-06T20:14:00.057310+0000","last_active":"2023-07-06T20:14:01.239597+0000","last_peered":"2023-07-06T20:14:01.239597+0000","last_clean":"2023-07-06T20:14:01.239597+0000","last_became_active":"2023-07-06T20:14:00.057048+0000","last_became_peered":"2023-07-06T20:14:00.057048+0000","last_unstale":"2023-07-06T20:14:01.239597+0000","last_undegraded":"2023-07-06T20:14:01.239597+0000","last_fullsized":"2023-07-06T20:14:01.239597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239261+0000","last_change":"2023-07-06T20:14:00.053189+0000","last_active":"2023-07-06T20:14:01.239261+0000","last_peered":"2023-07-06T20:14:01.239261+0000","last_clean":"2023-07-06T20:14:01.239261+0000","last_became_active":"2023-07-06T20:14:00.052649+0000","last_became_peered":"2023-07-06T20:14:00.052649+0000","last_unstale":"2023-07-06T20:14:01.239261+0000","last_undegraded":"2023-07-06T20:14:01.239261+0000","last_fullsized":"2023-07-06T20:14:01.239261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:44:22.477453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240134+0000","last_change":"2023-07-06T20:14:00.056320+0000","last_active":"2023-07-06T20:14:01.240134+0000","last_peered":"2023-07-06T20:14:01.240134+0000","last_clean":"2023-07-06T20:14:01.240134+0000","last_became_active":"2023-07-06T20:14:00.055864+0000","last_became_peered":"2023-07-06T20:14:00.055864+0000","last_unstale":"2023-07-06T20:14:01.240134+0000","last_undegraded":"2023-07-06T20:14:01.240134+0000","last_fullsized":"2023-07-06T20:14:01.240134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047499+0000","last_change":"2023-07-06T20:14:00.063394+0000","last_active":"2023-07-06T20:14:01.047499+0000","last_peered":"2023-07-06T20:14:01.047499+0000","last_clean":"2023-07-06T20:14:01.047499+0000","last_became_active":"2023-07-06T20:14:00.063126+0000","last_became_peered":"2023-07-06T20:14:00.063126+0000","last_unstale":"2023-07-06T20:14:01.047499+0000","last_undegraded":"2023-07-06T20:14:01.047499+0000","last_fullsized":"2023-07-06T20:14:01.047499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239136+0000","last_change":"2023-07-06T20:14:00.063773+0000","last_active":"2023-07-06T20:14:01.239136+0000","last_peered":"2023-07-06T20:14:01.239136+0000","last_clean":"2023-07-06T20:14:01.239136+0000","last_became_active":"2023-07-06T20:14:00.063498+0000","last_became_peered":"2023-07-06T20:14:00.063498+0000","last_unstale":"2023-07-06T20:14:01.239136+0000","last_undegraded":"2023-07-06T20:14:01.239136+0000","last_fullsized":"2023-07-06T20:14:01.239136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:24.723303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239028+0000","last_change":"2023-07-06T20:14:00.037831+0000","last_active":"2023-07-06T20:14:01.239028+0000","last_peered":"2023-07-06T20:14:01.239028+0000","last_clean":"2023-07-06T20:14:01.239028+0000","last_became_active":"2023-07-06T20:14:00.037512+0000","last_became_peered":"2023-07-06T20:14:00.037512+0000","last_unstale":"2023-07-06T20:14:01.239028+0000","last_undegraded":"2023-07-06T20:14:01.239028+0000","last_fullsized":"2023-07-06T20:14:01.239028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:17.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240002+0000","last_change":"2023-07-06T20:14:00.049995+0000","last_active":"2023-07-06T20:14:01.240002+0000","last_peered":"2023-07-06T20:14:01.240002+0000","last_clean":"2023-07-06T20:14:01.240002+0000","last_became_active":"2023-07-06T20:14:00.049548+0000","last_became_peered":"2023-07-06T20:14:00.049548+0000","last_unstale":"2023-07-06T20:14:01.240002+0000","last_undegraded":"2023-07-06T20:14:01.240002+0000","last_fullsized":"2023-07-06T20:14:01.240002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239729+0000","last_change":"2023-07-06T20:14:00.067827+0000","last_active":"2023-07-06T20:14:01.239729+0000","last_peered":"2023-07-06T20:14:01.239729+0000","last_clean":"2023-07-06T20:14:01.239729+0000","last_became_active":"2023-07-06T20:14:00.067219+0000","last_became_peered":"2023-07-06T20:14:00.067219+0000","last_unstale":"2023-07-06T20:14:01.239729+0000","last_undegraded":"2023-07-06T20:14:01.239729+0000","last_fullsized":"2023-07-06T20:14:01.239729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047086+0000","last_change":"2023-07-06T20:14:00.056312+0000","last_active":"2023-07-06T20:14:01.047086+0000","last_peered":"2023-07-06T20:14:01.047086+0000","last_clean":"2023-07-06T20:14:01.047086+0000","last_became_active":"2023-07-06T20:14:00.055971+0000","last_became_peered":"2023-07-06T20:14:00.055971+0000","last_unstale":"2023-07-06T20:14:01.047086+0000","last_undegraded":"2023-07-06T20:14:01.047086+0000","last_fullsized":"2023-07-06T20:14:01.047086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239406+0000","last_change":"2023-07-06T20:14:00.052988+0000","last_active":"2023-07-06T20:14:01.239406+0000","last_peered":"2023-07-06T20:14:01.239406+0000","last_clean":"2023-07-06T20:14:01.239406+0000","last_became_active":"2023-07-06T20:14:00.052324+0000","last_became_peered":"2023-07-06T20:14:00.052324+0000","last_unstale":"2023-07-06T20:14:01.239406+0000","last_undegraded":"2023-07-06T20:14:01.239406+0000","last_fullsized":"2023-07-06T20:14:01.239406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:32.142546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047643+0000","last_change":"2023-07-06T20:14:00.054064+0000","last_active":"2023-07-06T20:14:01.047643+0000","last_peered":"2023-07-06T20:14:01.047643+0000","last_clean":"2023-07-06T20:14:01.047643+0000","last_became_active":"2023-07-06T20:14:00.053459+0000","last_became_peered":"2023-07-06T20:14:00.053459+0000","last_unstale":"2023-07-06T20:14:01.047643+0000","last_undegraded":"2023-07-06T20:14:01.047643+0000","last_fullsized":"2023-07-06T20:14:01.047643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239803+0000","last_change":"2023-07-06T20:14:00.047280+0000","last_active":"2023-07-06T20:14:01.239803+0000","last_peered":"2023-07-06T20:14:01.239803+0000","last_clean":"2023-07-06T20:14:01.239803+0000","last_became_active":"2023-07-06T20:14:00.046943+0000","last_became_peered":"2023-07-06T20:14:00.046943+0000","last_unstale":"2023-07-06T20:14:01.239803+0000","last_undegraded":"2023-07-06T20:14:01.239803+0000","last_fullsized":"2023-07-06T20:14:01.239803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047192+0000","last_change":"2023-07-06T20:14:00.054317+0000","last_active":"2023-07-06T20:14:01.047192+0000","last_peered":"2023-07-06T20:14:01.047192+0000","last_clean":"2023-07-06T20:14:01.047192+0000","last_became_active":"2023-07-06T20:14:00.053672+0000","last_became_peered":"2023-07-06T20:14:00.053672+0000","last_unstale":"2023-07-06T20:14:01.047192+0000","last_undegraded":"2023-07-06T20:14:01.047192+0000","last_fullsized":"2023-07-06T20:14:01.047192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240189+0000","last_change":"2023-07-06T20:14:00.052959+0000","last_active":"2023-07-06T20:14:01.240189+0000","last_peered":"2023-07-06T20:14:01.240189+0000","last_clean":"2023-07-06T20:14:01.240189+0000","last_became_active":"2023-07-06T20:14:00.052605+0000","last_became_peered":"2023-07-06T20:14:00.052605+0000","last_unstale":"2023-07-06T20:14:01.240189+0000","last_undegraded":"2023-07-06T20:14:01.240189+0000","last_fullsized":"2023-07-06T20:14:01.240189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239515+0000","last_change":"2023-07-06T20:14:00.065147+0000","last_active":"2023-07-06T20:14:01.239515+0000","last_peered":"2023-07-06T20:14:01.239515+0000","last_clean":"2023-07-06T20:14:01.239515+0000","last_became_active":"2023-07-06T20:14:00.064652+0000","last_became_peered":"2023-07-06T20:14:00.064652+0000","last_unstale":"2023-07-06T20:14:01.239515+0000","last_undegraded":"2023-07-06T20:14:01.239515+0000","last_fullsized":"2023-07-06T20:14:01.239515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046578+0000","last_change":"2023-07-06T20:14:00.054578+0000","last_active":"2023-07-06T20:14:01.046578+0000","last_peered":"2023-07-06T20:14:01.046578+0000","last_clean":"2023-07-06T20:14:01.046578+0000","last_became_active":"2023-07-06T20:14:00.053689+0000","last_became_peered":"2023-07-06T20:14:00.053689+0000","last_unstale":"2023-07-06T20:14:01.046578+0000","last_undegraded":"2023-07-06T20:14:01.046578+0000","last_fullsized":"2023-07-06T20:14:01.046578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239287+0000","last_change":"2023-07-06T20:14:00.067375+0000","last_active":"2023-07-06T20:14:01.239287+0000","last_peered":"2023-07-06T20:14:01.239287+0000","last_clean":"2023-07-06T20:14:01.239287+0000","last_became_active":"2023-07-06T20:14:00.066909+0000","last_became_peered":"2023-07-06T20:14:00.066909+0000","last_unstale":"2023-07-06T20:14:01.239287+0000","last_undegraded":"2023-07-06T20:14:01.239287+0000","last_fullsized":"2023-07-06T20:14:01.239287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240385+0000","last_change":"2023-07-06T20:14:00.064820+0000","last_active":"2023-07-06T20:14:01.240385+0000","last_peered":"2023-07-06T20:14:01.240385+0000","last_clean":"2023-07-06T20:14:01.240385+0000","last_became_active":"2023-07-06T20:14:00.064585+0000","last_became_peered":"2023-07-06T20:14:00.064585+0000","last_unstale":"2023-07-06T20:14:01.240385+0000","last_undegraded":"2023-07-06T20:14:01.240385+0000","last_fullsized":"2023-07-06T20:14:01.240385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240518+0000","last_change":"2023-07-06T20:14:00.055061+0000","last_active":"2023-07-06T20:14:01.240518+0000","last_peered":"2023-07-06T20:14:01.240518+0000","last_clean":"2023-07-06T20:14:01.240518+0000","last_became_active":"2023-07-06T20:14:00.054795+0000","last_became_peered":"2023-07-06T20:14:00.054795+0000","last_unstale":"2023-07-06T20:14:01.240518+0000","last_undegraded":"2023-07-06T20:14:01.240518+0000","last_fullsized":"2023-07-06T20:14:01.240518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240667+0000","last_change":"2023-07-06T20:14:00.057631+0000","last_active":"2023-07-06T20:14:01.240667+0000","last_peered":"2023-07-06T20:14:01.240667+0000","last_clean":"2023-07-06T20:14:01.240667+0000","last_became_active":"2023-07-06T20:14:00.056582+0000","last_became_peered":"2023-07-06T20:14:00.056582+0000","last_unstale":"2023-07-06T20:14:01.240667+0000","last_undegraded":"2023-07-06T20:14:01.240667+0000","last_fullsized":"2023-07-06T20:14:01.240667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047284+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:14:01.047284+0000","last_peered":"2023-07-06T20:14:01.047284+0000","last_clean":"2023-07-06T20:14:01.047284+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:14:01.047284+0000","last_undegraded":"2023-07-06T20:14:01.047284+0000","last_fullsized":"2023-07-06T20:14:01.047284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047084+0000","last_change":"2023-07-06T20:14:00.067982+0000","last_active":"2023-07-06T20:14:01.047084+0000","last_peered":"2023-07-06T20:14:01.047084+0000","last_clean":"2023-07-06T20:14:01.047084+0000","last_became_active":"2023-07-06T20:14:00.067690+0000","last_became_peered":"2023-07-06T20:14:00.067690+0000","last_unstale":"2023-07-06T20:14:01.047084+0000","last_undegraded":"2023-07-06T20:14:01.047084+0000","last_fullsized":"2023-07-06T20:14:01.047084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239754+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:14:01.239754+0000","last_peered":"2023-07-06T20:14:01.239754+0000","last_clean":"2023-07-06T20:14:01.239754+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:14:01.239754+0000","last_undegraded":"2023-07-06T20:14:01.239754+0000","last_fullsized":"2023-07-06T20:14:01.239754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240204+0000","last_change":"2023-07-06T20:14:00.061405+0000","last_active":"2023-07-06T20:14:01.240204+0000","last_peered":"2023-07-06T20:14:01.240204+0000","last_clean":"2023-07-06T20:14:01.240204+0000","last_became_active":"2023-07-06T20:14:00.060981+0000","last_became_peered":"2023-07-06T20:14:00.060981+0000","last_unstale":"2023-07-06T20:14:01.240204+0000","last_undegraded":"2023-07-06T20:14:01.240204+0000","last_fullsized":"2023-07-06T20:14:01.240204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047136+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:14:01.047136+0000","last_peered":"2023-07-06T20:14:01.047136+0000","last_clean":"2023-07-06T20:14:01.047136+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:14:01.047136+0000","last_undegraded":"2023-07-06T20:14:01.047136+0000","last_fullsized":"2023-07-06T20:14:01.047136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239984+0000","last_change":"2023-07-06T20:14:00.052443+0000","last_active":"2023-07-06T20:14:01.239984+0000","last_peered":"2023-07-06T20:14:01.239984+0000","last_clean":"2023-07-06T20:14:01.239984+0000","last_became_active":"2023-07-06T20:14:00.052132+0000","last_became_peered":"2023-07-06T20:14:00.052132+0000","last_unstale":"2023-07-06T20:14:01.239984+0000","last_undegraded":"2023-07-06T20:14:01.239984+0000","last_fullsized":"2023-07-06T20:14:01.239984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:53.471708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239579+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:14:01.239579+0000","last_peered":"2023-07-06T20:14:01.239579+0000","last_clean":"2023-07-06T20:14:01.239579+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:14:01.239579+0000","last_undegraded":"2023-07-06T20:14:01.239579+0000","last_fullsized":"2023-07-06T20:14:01.239579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:14:00.067811+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:14:00.067528+0000","last_became_peered":"2023-07-06T20:14:00.067528+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239990+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:14:01.239990+0000","last_peered":"2023-07-06T20:14:01.239990+0000","last_clean":"2023-07-06T20:14:01.239990+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:14:01.239990+0000","last_undegraded":"2023-07-06T20:14:01.239990+0000","last_fullsized":"2023-07-06T20:14:01.239990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046539+0000","last_change":"2023-07-06T20:14:00.060824+0000","last_active":"2023-07-06T20:14:01.046539+0000","last_peered":"2023-07-06T20:14:01.046539+0000","last_clean":"2023-07-06T20:14:01.046539+0000","last_became_active":"2023-07-06T20:14:00.059809+0000","last_became_peered":"2023-07-06T20:14:00.059809+0000","last_unstale":"2023-07-06T20:14:01.046539+0000","last_undegraded":"2023-07-06T20:14:01.046539+0000","last_fullsized":"2023-07-06T20:14:01.046539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240120+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:14:01.240120+0000","last_peered":"2023-07-06T20:14:01.240120+0000","last_clean":"2023-07-06T20:14:01.240120+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.240120+0000","last_undegraded":"2023-07-06T20:14:01.240120+0000","last_fullsized":"2023-07-06T20:14:01.240120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:56.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240650+0000","last_change":"2023-07-06T20:14:00.056152+0000","last_active":"2023-07-06T20:14:01.240650+0000","last_peered":"2023-07-06T20:14:01.240650+0000","last_clean":"2023-07-06T20:14:01.240650+0000","last_became_active":"2023-07-06T20:14:00.055844+0000","last_became_peered":"2023-07-06T20:14:00.055844+0000","last_unstale":"2023-07-06T20:14:01.240650+0000","last_undegraded":"2023-07-06T20:14:01.240650+0000","last_fullsized":"2023-07-06T20:14:01.240650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047386+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:14:01.047386+0000","last_peered":"2023-07-06T20:14:01.047386+0000","last_clean":"2023-07-06T20:14:01.047386+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:14:01.047386+0000","last_undegraded":"2023-07-06T20:14:01.047386+0000","last_fullsized":"2023-07-06T20:14:01.047386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047231+0000","last_change":"2023-07-06T20:14:00.054939+0000","last_active":"2023-07-06T20:14:01.047231+0000","last_peered":"2023-07-06T20:14:01.047231+0000","last_clean":"2023-07-06T20:14:01.047231+0000","last_became_active":"2023-07-06T20:14:00.054354+0000","last_became_peered":"2023-07-06T20:14:00.054354+0000","last_unstale":"2023-07-06T20:14:01.047231+0000","last_undegraded":"2023-07-06T20:14:01.047231+0000","last_fullsized":"2023-07-06T20:14:01.047231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240033+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:14:01.240033+0000","last_peered":"2023-07-06T20:14:01.240033+0000","last_clean":"2023-07-06T20:14:01.240033+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:14:01.240033+0000","last_undegraded":"2023-07-06T20:14:01.240033+0000","last_fullsized":"2023-07-06T20:14:01.240033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:26.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046831+0000","last_change":"2023-07-06T20:14:00.056179+0000","last_active":"2023-07-06T20:14:01.046831+0000","last_peered":"2023-07-06T20:14:01.046831+0000","last_clean":"2023-07-06T20:14:01.046831+0000","last_became_active":"2023-07-06T20:14:00.055858+0000","last_became_peered":"2023-07-06T20:14:00.055858+0000","last_unstale":"2023-07-06T20:14:01.046831+0000","last_undegraded":"2023-07-06T20:14:01.046831+0000","last_fullsized":"2023-07-06T20:14:01.046831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239368+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:14:01.239368+0000","last_peered":"2023-07-06T20:14:01.239368+0000","last_clean":"2023-07-06T20:14:01.239368+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:14:01.239368+0000","last_undegraded":"2023-07-06T20:14:01.239368+0000","last_fullsized":"2023-07-06T20:14:01.239368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:48.108641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240807+0000","last_change":"2023-07-06T20:14:00.056298+0000","last_active":"2023-07-06T20:14:01.240807+0000","last_peered":"2023-07-06T20:14:01.240807+0000","last_clean":"2023-07-06T20:14:01.240807+0000","last_became_active":"2023-07-06T20:14:00.055932+0000","last_became_peered":"2023-07-06T20:14:00.055932+0000","last_unstale":"2023-07-06T20:14:01.240807+0000","last_undegraded":"2023-07-06T20:14:01.240807+0000","last_fullsized":"2023-07-06T20:14:01.240807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239561+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:14:01.239561+0000","last_peered":"2023-07-06T20:14:01.239561+0000","last_clean":"2023-07-06T20:14:01.239561+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:14:01.239561+0000","last_undegraded":"2023-07-06T20:14:01.239561+0000","last_fullsized":"2023-07-06T20:14:01.239561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240113+0000","last_change":"2023-07-06T20:14:00.064964+0000","last_active":"2023-07-06T20:14:01.240113+0000","last_peered":"2023-07-06T20:14:01.240113+0000","last_clean":"2023-07-06T20:14:01.240113+0000","last_became_active":"2023-07-06T20:14:00.064266+0000","last_became_peered":"2023-07-06T20:14:00.064266+0000","last_unstale":"2023-07-06T20:14:01.240113+0000","last_undegraded":"2023-07-06T20:14:01.240113+0000","last_fullsized":"2023-07-06T20:14:01.240113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239123+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:14:01.239123+0000","last_peered":"2023-07-06T20:14:01.239123+0000","last_clean":"2023-07-06T20:14:01.239123+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:14:01.239123+0000","last_undegraded":"2023-07-06T20:14:01.239123+0000","last_fullsized":"2023-07-06T20:14:01.239123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047315+0000","last_change":"2023-07-06T20:14:00.061173+0000","last_active":"2023-07-06T20:14:01.047315+0000","last_peered":"2023-07-06T20:14:01.047315+0000","last_clean":"2023-07-06T20:14:01.047315+0000","last_became_active":"2023-07-06T20:14:00.060346+0000","last_became_peered":"2023-07-06T20:14:00.060346+0000","last_unstale":"2023-07-06T20:14:01.047315+0000","last_undegraded":"2023-07-06T20:14:01.047315+0000","last_fullsized":"2023-07-06T20:14:01.047315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239999+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:14:01.239999+0000","last_peered":"2023-07-06T20:14:01.239999+0000","last_clean":"2023-07-06T20:14:01.239999+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:14:01.239999+0000","last_undegraded":"2023-07-06T20:14:01.239999+0000","last_fullsized":"2023-07-06T20:14:01.239999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:37.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239775+0000","last_change":"2023-07-06T20:14:00.061044+0000","last_active":"2023-07-06T20:14:01.239775+0000","last_peered":"2023-07-06T20:14:01.239775+0000","last_clean":"2023-07-06T20:14:01.239775+0000","last_became_active":"2023-07-06T20:14:00.060737+0000","last_became_peered":"2023-07-06T20:14:00.060737+0000","last_unstale":"2023-07-06T20:14:01.239775+0000","last_undegraded":"2023-07-06T20:14:01.239775+0000","last_fullsized":"2023-07-06T20:14:01.239775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240322+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:14:01.240322+0000","last_peered":"2023-07-06T20:14:01.240322+0000","last_clean":"2023-07-06T20:14:01.240322+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:14:01.240322+0000","last_undegraded":"2023-07-06T20:14:01.240322+0000","last_fullsized":"2023-07-06T20:14:01.240322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.062318+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.062063+0000","last_became_peered":"2023-07-06T20:14:00.062063+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240374+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:14:01.240374+0000","last_peered":"2023-07-06T20:14:01.240374+0000","last_clean":"2023-07-06T20:14:01.240374+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:14:01.240374+0000","last_undegraded":"2023-07-06T20:14:01.240374+0000","last_fullsized":"2023-07-06T20:14:01.240374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:11.129495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240536+0000","last_change":"2023-07-06T20:14:00.064988+0000","last_active":"2023-07-06T20:14:01.240536+0000","last_peered":"2023-07-06T20:14:01.240536+0000","last_clean":"2023-07-06T20:14:01.240536+0000","last_became_active":"2023-07-06T20:14:00.064581+0000","last_became_peered":"2023-07-06T20:14:00.064581+0000","last_unstale":"2023-07-06T20:14:01.240536+0000","last_undegraded":"2023-07-06T20:14:01.240536+0000","last_fullsized":"2023-07-06T20:14:01.240536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:10.399206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239427+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:14:01.239427+0000","last_peered":"2023-07-06T20:14:01.239427+0000","last_clean":"2023-07-06T20:14:01.239427+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:14:01.239427+0000","last_undegraded":"2023-07-06T20:14:01.239427+0000","last_fullsized":"2023-07-06T20:14:01.239427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:18.117668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239302+0000","last_change":"2023-07-06T20:14:00.065729+0000","last_active":"2023-07-06T20:14:01.239302+0000","last_peered":"2023-07-06T20:14:01.239302+0000","last_clean":"2023-07-06T20:14:01.239302+0000","last_became_active":"2023-07-06T20:14:00.064827+0000","last_became_peered":"2023-07-06T20:14:00.064827+0000","last_unstale":"2023-07-06T20:14:01.239302+0000","last_undegraded":"2023-07-06T20:14:01.239302+0000","last_fullsized":"2023-07-06T20:14:01.239302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:21:48.022430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046583+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:14:01.046583+0000","last_peered":"2023-07-06T20:14:01.046583+0000","last_clean":"2023-07-06T20:14:01.046583+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.046583+0000","last_undegraded":"2023-07-06T20:14:01.046583+0000","last_fullsized":"2023-07-06T20:14:01.046583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:18.188230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047260+0000","last_change":"2023-07-06T20:14:00.054542+0000","last_active":"2023-07-06T20:14:01.047260+0000","last_peered":"2023-07-06T20:14:01.047260+0000","last_clean":"2023-07-06T20:14:01.047260+0000","last_became_active":"2023-07-06T20:14:00.053987+0000","last_became_peered":"2023-07-06T20:14:00.053987+0000","last_unstale":"2023-07-06T20:14:01.047260+0000","last_undegraded":"2023-07-06T20:14:01.047260+0000","last_fullsized":"2023-07-06T20:14:01.047260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240427+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:14:01.240427+0000","last_peered":"2023-07-06T20:14:01.240427+0000","last_clean":"2023-07-06T20:14:01.240427+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:14:01.240427+0000","last_undegraded":"2023-07-06T20:14:01.240427+0000","last_fullsized":"2023-07-06T20:14:01.240427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:32.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047708+0000","last_change":"2023-07-06T20:14:00.052728+0000","last_active":"2023-07-06T20:14:01.047708+0000","last_peered":"2023-07-06T20:14:01.047708+0000","last_clean":"2023-07-06T20:14:01.047708+0000","last_became_active":"2023-07-06T20:14:00.052437+0000","last_became_peered":"2023-07-06T20:14:00.052437+0000","last_unstale":"2023-07-06T20:14:01.047708+0000","last_undegraded":"2023-07-06T20:14:01.047708+0000","last_fullsized":"2023-07-06T20:14:01.047708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:27.783430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047897+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:14:01.047897+0000","last_peered":"2023-07-06T20:14:01.047897+0000","last_clean":"2023-07-06T20:14:01.047897+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:14:01.047897+0000","last_undegraded":"2023-07-06T20:14:01.047897+0000","last_fullsized":"2023-07-06T20:14:01.047897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.053789+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.053148+0000","last_became_peered":"2023-07-06T20:14:00.053148+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:57.270521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046721+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:14:01.046721+0000","last_peered":"2023-07-06T20:14:01.046721+0000","last_clean":"2023-07-06T20:14:01.046721+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:14:01.046721+0000","last_undegraded":"2023-07-06T20:14:01.046721+0000","last_fullsized":"2023-07-06T20:14:01.046721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240548+0000","last_change":"2023-07-06T20:14:00.056310+0000","last_active":"2023-07-06T20:14:01.240548+0000","last_peered":"2023-07-06T20:14:01.240548+0000","last_clean":"2023-07-06T20:14:01.240548+0000","last_became_active":"2023-07-06T20:14:00.055916+0000","last_became_peered":"2023-07-06T20:14:00.055916+0000","last_unstale":"2023-07-06T20:14:01.240548+0000","last_undegraded":"2023-07-06T20:14:01.240548+0000","last_fullsized":"2023-07-06T20:14:01.240548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239879+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:14:01.239879+0000","last_peered":"2023-07-06T20:14:01.239879+0000","last_clean":"2023-07-06T20:14:01.239879+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:14:01.239879+0000","last_undegraded":"2023-07-06T20:14:01.239879+0000","last_fullsized":"2023-07-06T20:14:01.239879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:12.079947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240842+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:14:01.240842+0000","last_peered":"2023-07-06T20:14:01.240842+0000","last_clean":"2023-07-06T20:14:01.240842+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:14:01.240842+0000","last_undegraded":"2023-07-06T20:14:01.240842+0000","last_fullsized":"2023-07-06T20:14:01.240842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240741+0000","last_change":"2023-07-06T20:14:00.055724+0000","last_active":"2023-07-06T20:14:01.240741+0000","last_peered":"2023-07-06T20:14:01.240741+0000","last_clean":"2023-07-06T20:14:01.240741+0000","last_became_active":"2023-07-06T20:14:00.055440+0000","last_became_peered":"2023-07-06T20:14:00.055440+0000","last_unstale":"2023-07-06T20:14:01.240741+0000","last_undegraded":"2023-07-06T20:14:01.240741+0000","last_fullsized":"2023-07-06T20:14:01.240741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240130+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:14:01.240130+0000","last_peered":"2023-07-06T20:14:01.240130+0000","last_clean":"2023-07-06T20:14:01.240130+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:14:01.240130+0000","last_undegraded":"2023-07-06T20:14:01.240130+0000","last_fullsized":"2023-07-06T20:14:01.240130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:50.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047412+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:14:01.047412+0000","last_peered":"2023-07-06T20:14:01.047412+0000","last_clean":"2023-07-06T20:14:01.047412+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:14:01.047412+0000","last_undegraded":"2023-07-06T20:14:01.047412+0000","last_fullsized":"2023-07-06T20:14:01.047412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:17.962306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240019+0000","last_change":"2023-07-06T20:14:00.055529+0000","last_active":"2023-07-06T20:14:01.240019+0000","last_peered":"2023-07-06T20:14:01.240019+0000","last_clean":"2023-07-06T20:14:01.240019+0000","last_became_active":"2023-07-06T20:14:00.054757+0000","last_became_peered":"2023-07-06T20:14:00.054757+0000","last_unstale":"2023-07-06T20:14:01.240019+0000","last_undegraded":"2023-07-06T20:14:01.240019+0000","last_fullsized":"2023-07-06T20:14:01.240019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238739+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:14:01.238739+0000","last_peered":"2023-07-06T20:14:01.238739+0000","last_clean":"2023-07-06T20:14:01.238739+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:14:01.238739+0000","last_undegraded":"2023-07-06T20:14:01.238739+0000","last_fullsized":"2023-07-06T20:14:01.238739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239367+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:14:01.239367+0000","last_peered":"2023-07-06T20:14:01.239367+0000","last_clean":"2023-07-06T20:14:01.239367+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:14:01.239367+0000","last_undegraded":"2023-07-06T20:14:01.239367+0000","last_fullsized":"2023-07-06T20:14:01.239367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240403+0000","last_change":"2023-07-06T20:14:00.062155+0000","last_active":"2023-07-06T20:14:01.240403+0000","last_peered":"2023-07-06T20:14:01.240403+0000","last_clean":"2023-07-06T20:14:01.240403+0000","last_became_active":"2023-07-06T20:14:00.061848+0000","last_became_peered":"2023-07-06T20:14:00.061848+0000","last_unstale":"2023-07-06T20:14:01.240403+0000","last_undegraded":"2023-07-06T20:14:01.240403+0000","last_fullsized":"2023-07-06T20:14:01.240403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239665+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:14:01.239665+0000","last_peered":"2023-07-06T20:14:01.239665+0000","last_clean":"2023-07-06T20:14:01.239665+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:14:01.239665+0000","last_undegraded":"2023-07-06T20:14:01.239665+0000","last_fullsized":"2023-07-06T20:14:01.239665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:53.676445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239234+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:14:01.239234+0000","last_peered":"2023-07-06T20:14:01.239234+0000","last_clean":"2023-07-06T20:14:01.239234+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:14:01.239234+0000","last_undegraded":"2023-07-06T20:14:01.239234+0000","last_fullsized":"2023-07-06T20:14:01.239234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:04.461531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046254+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:14:01.046254+0000","last_peered":"2023-07-06T20:14:01.046254+0000","last_clean":"2023-07-06T20:14:01.046254+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:14:01.046254+0000","last_undegraded":"2023-07-06T20:14:01.046254+0000","last_fullsized":"2023-07-06T20:14:01.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239372+0000","last_change":"2023-07-06T20:14:00.062536+0000","last_active":"2023-07-06T20:14:01.239372+0000","last_peered":"2023-07-06T20:14:01.239372+0000","last_clean":"2023-07-06T20:14:01.239372+0000","last_became_active":"2023-07-06T20:14:00.062189+0000","last_became_peered":"2023-07-06T20:14:00.062189+0000","last_unstale":"2023-07-06T20:14:01.239372+0000","last_undegraded":"2023-07-06T20:14:01.239372+0000","last_fullsized":"2023-07-06T20:14:01.239372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:07.175614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239383+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:14:01.239383+0000","last_peered":"2023-07-06T20:14:01.239383+0000","last_clean":"2023-07-06T20:14:01.239383+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:14:01.239383+0000","last_undegraded":"2023-07-06T20:14:01.239383+0000","last_fullsized":"2023-07-06T20:14:01.239383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:38.150080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239126+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:14:01.239126+0000","last_peered":"2023-07-06T20:14:01.239126+0000","last_clean":"2023-07-06T20:14:01.239126+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:14:01.239126+0000","last_undegraded":"2023-07-06T20:14:01.239126+0000","last_fullsized":"2023-07-06T20:14:01.239126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:42.107407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240485+0000","last_change":"2023-07-06T20:14:00.065377+0000","last_active":"2023-07-06T20:14:01.240485+0000","last_peered":"2023-07-06T20:14:01.240485+0000","last_clean":"2023-07-06T20:14:01.240485+0000","last_became_active":"2023-07-06T20:14:00.064627+0000","last_became_peered":"2023-07-06T20:14:00.064627+0000","last_unstale":"2023-07-06T20:14:01.240485+0000","last_undegraded":"2023-07-06T20:14:01.240485+0000","last_fullsized":"2023-07-06T20:14:01.240485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240293+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:14:01.240293+0000","last_peered":"2023-07-06T20:14:01.240293+0000","last_clean":"2023-07-06T20:14:01.240293+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:14:01.240293+0000","last_undegraded":"2023-07-06T20:14:01.240293+0000","last_fullsized":"2023-07-06T20:14:01.240293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:41:59.625038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240045+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:14:01.240045+0000","last_peered":"2023-07-06T20:14:01.240045+0000","last_clean":"2023-07-06T20:14:01.240045+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:14:01.240045+0000","last_undegraded":"2023-07-06T20:14:01.240045+0000","last_fullsized":"2023-07-06T20:14:01.240045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239757+0000","last_change":"2023-07-06T20:14:00.057341+0000","last_active":"2023-07-06T20:14:01.239757+0000","last_peered":"2023-07-06T20:14:01.239757+0000","last_clean":"2023-07-06T20:14:01.239757+0000","last_became_active":"2023-07-06T20:14:00.057077+0000","last_became_peered":"2023-07-06T20:14:00.057077+0000","last_unstale":"2023-07-06T20:14:01.239757+0000","last_undegraded":"2023-07-06T20:14:01.239757+0000","last_fullsized":"2023-07-06T20:14:01.239757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047029+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:14:01.047029+0000","last_peered":"2023-07-06T20:14:01.047029+0000","last_clean":"2023-07-06T20:14:01.047029+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:14:01.047029+0000","last_undegraded":"2023-07-06T20:14:01.047029+0000","last_fullsized":"2023-07-06T20:14:01.047029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:37.193187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240442+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:14:01.240442+0000","last_peered":"2023-07-06T20:14:01.240442+0000","last_clean":"2023-07-06T20:14:01.240442+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:14:01.240442+0000","last_undegraded":"2023-07-06T20:14:01.240442+0000","last_fullsized":"2023-07-06T20:14:01.240442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:03.076475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046144+0000","last_change":"2023-07-06T20:14:00.066131+0000","last_active":"2023-07-06T20:14:01.046144+0000","last_peered":"2023-07-06T20:14:01.046144+0000","last_clean":"2023-07-06T20:14:01.046144+0000","last_became_active":"2023-07-06T20:14:00.065793+0000","last_became_peered":"2023-07-06T20:14:00.065793+0000","last_unstale":"2023-07-06T20:14:01.046144+0000","last_undegraded":"2023-07-06T20:14:01.046144+0000","last_fullsized":"2023-07-06T20:14:01.046144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:49.111646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239813+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:14:01.239813+0000","last_peered":"2023-07-06T20:14:01.239813+0000","last_clean":"2023-07-06T20:14:01.239813+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:14:01.239813+0000","last_undegraded":"2023-07-06T20:14:01.239813+0000","last_fullsized":"2023-07-06T20:14:01.239813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:17.841824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240384+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:14:01.240384+0000","last_peered":"2023-07-06T20:14:01.240384+0000","last_clean":"2023-07-06T20:14:01.240384+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:14:01.240384+0000","last_undegraded":"2023-07-06T20:14:01.240384+0000","last_fullsized":"2023-07-06T20:14:01.240384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047806+0000","last_change":"2023-07-06T20:14:00.055463+0000","last_active":"2023-07-06T20:14:01.047806+0000","last_peered":"2023-07-06T20:14:01.047806+0000","last_clean":"2023-07-06T20:14:01.047806+0000","last_became_active":"2023-07-06T20:14:00.055196+0000","last_became_peered":"2023-07-06T20:14:00.055196+0000","last_unstale":"2023-07-06T20:14:01.047806+0000","last_undegraded":"2023-07-06T20:14:01.047806+0000","last_fullsized":"2023-07-06T20:14:01.047806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:42:06.665764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239438+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:14:01.239438+0000","last_peered":"2023-07-06T20:14:01.239438+0000","last_clean":"2023-07-06T20:14:01.239438+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:14:01.239438+0000","last_undegraded":"2023-07-06T20:14:01.239438+0000","last_fullsized":"2023-07-06T20:14:01.239438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239401+0000","last_change":"2023-07-06T20:14:00.067006+0000","last_active":"2023-07-06T20:14:01.239401+0000","last_peered":"2023-07-06T20:14:01.239401+0000","last_clean":"2023-07-06T20:14:01.239401+0000","last_became_active":"2023-07-06T20:14:00.066735+0000","last_became_peered":"2023-07-06T20:14:00.066735+0000","last_unstale":"2023-07-06T20:14:01.239401+0000","last_undegraded":"2023-07-06T20:14:01.239401+0000","last_fullsized":"2023-07-06T20:14:01.239401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:37.878199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239281+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:14:01.239281+0000","last_peered":"2023-07-06T20:14:01.239281+0000","last_clean":"2023-07-06T20:14:01.239281+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:14:01.239281+0000","last_undegraded":"2023-07-06T20:14:01.239281+0000","last_fullsized":"2023-07-06T20:14:01.239281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047642+0000","last_change":"2023-07-06T20:14:00.054948+0000","last_active":"2023-07-06T20:14:01.047642+0000","last_peered":"2023-07-06T20:14:01.047642+0000","last_clean":"2023-07-06T20:14:01.047642+0000","last_became_active":"2023-07-06T20:14:00.054677+0000","last_became_peered":"2023-07-06T20:14:00.054677+0000","last_unstale":"2023-07-06T20:14:01.047642+0000","last_undegraded":"2023-07-06T20:14:01.047642+0000","last_fullsized":"2023-07-06T20:14:01.047642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:38.225314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240958+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:14:01.240958+0000","last_peered":"2023-07-06T20:14:01.240958+0000","last_clean":"2023-07-06T20:14:01.240958+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:14:01.240958+0000","last_undegraded":"2023-07-06T20:14:01.240958+0000","last_fullsized":"2023-07-06T20:14:01.240958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:37.736933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047478+0000","last_change":"2023-07-06T20:14:00.053372+0000","last_active":"2023-07-06T20:14:01.047478+0000","last_peered":"2023-07-06T20:14:01.047478+0000","last_clean":"2023-07-06T20:14:01.047478+0000","last_became_active":"2023-07-06T20:14:00.053043+0000","last_became_peered":"2023-07-06T20:14:00.053043+0000","last_unstale":"2023-07-06T20:14:01.047478+0000","last_undegraded":"2023-07-06T20:14:01.047478+0000","last_fullsized":"2023-07-06T20:14:01.047478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046413+0000","last_change":"2023-07-06T20:14:00.038980+0000","last_active":"2023-07-06T20:14:01.046413+0000","last_peered":"2023-07-06T20:14:01.046413+0000","last_clean":"2023-07-06T20:14:01.046413+0000","last_became_active":"2023-07-06T20:14:00.038706+0000","last_became_peered":"2023-07-06T20:14:00.038706+0000","last_unstale":"2023-07-06T20:14:01.046413+0000","last_undegraded":"2023-07-06T20:14:01.046413+0000","last_fullsized":"2023-07-06T20:14:01.046413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239873+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:14:01.239873+0000","last_peered":"2023-07-06T20:14:01.239873+0000","last_clean":"2023-07-06T20:14:01.239873+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:14:01.239873+0000","last_undegraded":"2023-07-06T20:14:01.239873+0000","last_fullsized":"2023-07-06T20:14:01.239873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047021+0000","last_change":"2023-07-06T20:14:00.049540+0000","last_active":"2023-07-06T20:14:01.047021+0000","last_peered":"2023-07-06T20:14:01.047021+0000","last_clean":"2023-07-06T20:14:01.047021+0000","last_became_active":"2023-07-06T20:14:00.049260+0000","last_became_peered":"2023-07-06T20:14:00.049260+0000","last_unstale":"2023-07-06T20:14:01.047021+0000","last_undegraded":"2023-07-06T20:14:01.047021+0000","last_fullsized":"2023-07-06T20:14:01.047021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239288+0000","last_change":"2023-07-06T20:14:00.048736+0000","last_active":"2023-07-06T20:14:01.239288+0000","last_peered":"2023-07-06T20:14:01.239288+0000","last_clean":"2023-07-06T20:14:01.239288+0000","last_became_active":"2023-07-06T20:14:00.048418+0000","last_became_peered":"2023-07-06T20:14:00.048418+0000","last_unstale":"2023-07-06T20:14:01.239288+0000","last_undegraded":"2023-07-06T20:14:01.239288+0000","last_fullsized":"2023-07-06T20:14:01.239288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046375+0000","last_change":"2023-07-06T20:14:00.067909+0000","last_active":"2023-07-06T20:14:01.046375+0000","last_peered":"2023-07-06T20:14:01.046375+0000","last_clean":"2023-07-06T20:14:01.046375+0000","last_became_active":"2023-07-06T20:14:00.067600+0000","last_became_peered":"2023-07-06T20:14:00.067600+0000","last_unstale":"2023-07-06T20:14:01.046375+0000","last_undegraded":"2023-07-06T20:14:01.046375+0000","last_fullsized":"2023-07-06T20:14:01.046375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047258+0000","last_change":"2023-07-06T20:14:00.058103+0000","last_active":"2023-07-06T20:14:01.047258+0000","last_peered":"2023-07-06T20:14:01.047258+0000","last_clean":"2023-07-06T20:14:01.047258+0000","last_became_active":"2023-07-06T20:14:00.057800+0000","last_became_peered":"2023-07-06T20:14:00.057800+0000","last_unstale":"2023-07-06T20:14:01.047258+0000","last_undegraded":"2023-07-06T20:14:01.047258+0000","last_fullsized":"2023-07-06T20:14:01.047258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046528+0000","last_change":"2023-07-06T20:14:00.061079+0000","last_active":"2023-07-06T20:14:01.046528+0000","last_peered":"2023-07-06T20:14:01.046528+0000","last_clean":"2023-07-06T20:14:01.046528+0000","last_became_active":"2023-07-06T20:14:00.059768+0000","last_became_peered":"2023-07-06T20:14:00.059768+0000","last_unstale":"2023-07-06T20:14:01.046528+0000","last_undegraded":"2023-07-06T20:14:01.046528+0000","last_fullsized":"2023-07-06T20:14:01.046528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239855+0000","last_change":"2023-07-06T20:14:00.056153+0000","last_active":"2023-07-06T20:14:01.239855+0000","last_peered":"2023-07-06T20:14:01.239855+0000","last_clean":"2023-07-06T20:14:01.239855+0000","last_became_active":"2023-07-06T20:14:00.055830+0000","last_became_peered":"2023-07-06T20:14:00.055830+0000","last_unstale":"2023-07-06T20:14:01.239855+0000","last_undegraded":"2023-07-06T20:14:01.239855+0000","last_fullsized":"2023-07-06T20:14:01.239855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:09.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:09.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.712+0000 7f4623fff700 1 Processor -- start 2023-07-06T20:14:09.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.712+0000 7f4623fff700 1 -- start start 2023-07-06T20:14:09.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.713+0000 7f4623fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:09.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.713+0000 7f4623fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0cf780 con 0x7f461c0b5c90 2023-07-06T20:14:09.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.713+0000 7f4623fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c0cd490 con 0x7f461c15fca0 2023-07-06T20:14:09.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.713+0000 7f4623fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cd770 con 0x7f461c0b6110 2023-07-06T20:14:09.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f462159a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f461c0b5c90 0x7f461c0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46104/0 (socket says 172.21.15.27:46104) 2023-07-06T20:14:09.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f462159a700 1 -- 172.21.15.27:0/3989663279 learned_addr learned my addr 172.21.15.27:0/3989663279 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f462259c700 1 --2- 172.21.15.27:0/3989663279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412532581 0 0) 0x7f461c0cf780 con 0x7f461c0b5c90 2023-07-06T20:14:09.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46080035b0 con 0x7f461c0b5c90 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100664032 0 0) 0x7f461c0cd490 con 0x7f461c15fca0 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f461c0cf780 con 0x7f461c15fca0 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f462259c700 1 -- 172.21.15.27:0/3989663279 >> v1:172.21.15.153:6789/0 conn(0x7f461c15fca0 legacy=0x7f461c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f462259c700 1 -- 172.21.15.27:0/3989663279 >> v1:172.21.15.27:6789/0 conn(0x7f461c0b5c90 legacy=0x7f461c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.714+0000 7f462259c700 1 -- 172.21.15.27:0/3989663279 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0cda50 con 0x7f461c0b6110 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.715+0000 7f462259c700 1 --2- 172.21.15.27:0/3989663279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c0cfc10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4614009dc0 tx=0x7f461400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe5980d752da13e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.715+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400e070 con 0x7f461c0b6110 2023-07-06T20:14:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.715+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461400bb20 con 0x7f461c0b6110 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.715+0000 7f4620d99700 1 -- 172.21.15.27:0/3989663279 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614012430 con 0x7f461c0b6110 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.715+0000 7f4623fff700 1 -- 172.21.15.27:0/3989663279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 msgr2=0x7f461c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.715+0000 7f4623fff700 1 --2- 172.21.15.27:0/3989663279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c0cfc10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4614009dc0 tx=0x7f461400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.716+0000 7f4623fff700 1 -- 172.21.15.27:0/3989663279 shutdown_connections 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.716+0000 7f4623fff700 1 --2- 172.21.15.27:0/3989663279 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c0cfc10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.716+0000 7f4623fff700 1 -- 172.21.15.27:0/3989663279 >> 172.21.15.27:0/3989663279 conn(0x7f461c06c150 msgr2=0x7f461c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.716+0000 7f4623fff700 1 -- 172.21.15.27:0/3989663279 shutdown_connections 2023-07-06T20:14:09.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.716+0000 7f4623fff700 1 -- 172.21.15.27:0/3989663279 wait complete. 2023-07-06T20:14:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.717+0000 7f4623fff700 1 Processor -- start 2023-07-06T20:14:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.718+0000 7f4623fff700 1 -- start start 2023-07-06T20:14:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.718+0000 7f4623fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 0x7f461c1e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.718+0000 7f4623fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c13c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4623fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461c15fca0 0x7f461c13cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4623fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cbae0 con 0x7f461c0b5c90 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4623fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0caed0 con 0x7f461c15fca0 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4623fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cb1b0 con 0x7f461c0b6110 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 0x7f461c1e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 0x7f461c1e5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59274/0 (socket says 172.21.15.27:59274) 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 -- 172.21.15.27:0/2633969393 learned_addr learned my addr 172.21.15.27:0/2633969393 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f462159a700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c13c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f462259c700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461c15fca0 0x7f461c13cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 -- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 msgr2=0x7f461c13c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c13c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 -- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461c15fca0 msgr2=0x7f461c13cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461c15fca0 0x7f461c13cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 -- 172.21.15.27:0/2633969393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009a70 con 0x7f461c0b5c90 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f462259c700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461c15fca0 0x7f461c13cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.719+0000 7f4621d9b700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 0x7f461c1e5ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f461800c500 tx=0x7f461800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.720+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180052d0 con 0x7f461c0b5c90 2023-07-06T20:14:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.720+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618005450 con 0x7f461c0b5c90 2023-07-06T20:14:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.720+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180038c0 con 0x7f461c0b5c90 2023-07-06T20:14:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.720+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0cb490 con 0x7f461c0b5c90 2023-07-06T20:14:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.720+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c1e8f60 con 0x7f461c0b5c90 2023-07-06T20:14:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.721+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c0cbdc0 con 0x7f461c0b5c90 2023-07-06T20:14:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.722+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4618003a20 con 0x7f461c0b5c90 2023-07-06T20:14:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.723+0000 7f46127fc700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f460006ace0 0x7f460006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.723+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4618088950 con 0x7f461c0b5c90 2023-07-06T20:14:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.723+0000 7f462159a700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f460006ace0 0x7f460006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.724+0000 7f462159a700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f460006ace0 0x7f460006d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f461c13db00 tx=0x7f460c007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:09.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.727+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4618053960 con 0x7f461c0b5c90 2023-07-06T20:14:09.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.929+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f461c06ae40 con 0x7f460006ace0 2023-07-06T20:14:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.938+0000 7f46127fc700 1 -- 172.21.15.27:0/2633969393 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286739 (secure 0 0 0) 0x7f460c0058e0 con 0x7f460006ace0 2023-07-06T20:14:09.943 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f460006ace0 msgr2=0x7f460006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f460006ace0 0x7f460006d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f461c13db00 tx=0x7f460c007dc0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 msgr2=0x7f461c1e5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 0x7f461c1e5ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f461800c500 tx=0x7f461800c810 comp rx=0 tx=0).stop 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 shutdown_connections 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f460006ace0 0x7f460006d190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461c15fca0 0x7f461c13cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f461c0b5c90 0x7f461c1e5ee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 --2- 172.21.15.27:0/2633969393 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f461c0b6110 0x7f461c13c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 >> 172.21.15.27:0/2633969393 conn(0x7f461c06c150 msgr2=0x7f461c0b8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 shutdown_connections 2023-07-06T20:14:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:09.942+0000 7f4623fff700 1 -- 172.21.15.27:0/2633969393 wait complete. 2023-07-06T20:14:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:09.962 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-07-06T20:14:08.344829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.319209"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238965+0000","last_change":"2023-07-06T20:14:00.041890+0000","last_active":"2023-07-06T20:14:01.238965+0000","last_peered":"2023-07-06T20:14:01.238965+0000","last_clean":"2023-07-06T20:14:01.238965+0000","last_became_active":"2023-07-06T20:14:00.041518+0000","last_became_peered":"2023-07-06T20:14:00.041518+0000","last_unstale":"2023-07-06T20:14:01.238965+0000","last_undegraded":"2023-07-06T20:14:01.238965+0000","last_fullsized":"2023-07-06T20:14:01.238965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:26.527907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239844+0000","last_change":"2023-07-06T20:14:00.067412+0000","last_active":"2023-07-06T20:14:01.239844+0000","last_peered":"2023-07-06T20:14:01.239844+0000","last_clean":"2023-07-06T20:14:01.239844+0000","last_became_active":"2023-07-06T20:14:00.067104+0000","last_became_peered":"2023-07-06T20:14:00.067104+0000","last_unstale":"2023-07-06T20:14:01.239844+0000","last_undegraded":"2023-07-06T20:14:01.239844+0000","last_fullsized":"2023-07-06T20:14:01.239844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047383+0000","last_change":"2023-07-06T20:14:00.056588+0000","last_active":"2023-07-06T20:14:01.047383+0000","last_peered":"2023-07-06T20:14:01.047383+0000","last_clean":"2023-07-06T20:14:01.047383+0000","last_became_active":"2023-07-06T20:14:00.056267+0000","last_became_peered":"2023-07-06T20:14:00.056267+0000","last_unstale":"2023-07-06T20:14:01.047383+0000","last_undegraded":"2023-07-06T20:14:01.047383+0000","last_fullsized":"2023-07-06T20:14:01.047383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240208+0000","last_change":"2023-07-06T20:14:00.058980+0000","last_active":"2023-07-06T20:14:01.240208+0000","last_peered":"2023-07-06T20:14:01.240208+0000","last_clean":"2023-07-06T20:14:01.240208+0000","last_became_active":"2023-07-06T20:14:00.058707+0000","last_became_peered":"2023-07-06T20:14:00.058707+0000","last_unstale":"2023-07-06T20:14:01.240208+0000","last_undegraded":"2023-07-06T20:14:01.240208+0000","last_fullsized":"2023-07-06T20:14:01.240208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:14.285291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239684+0000","last_change":"2023-07-06T20:14:00.064519+0000","last_active":"2023-07-06T20:14:01.239684+0000","last_peered":"2023-07-06T20:14:01.239684+0000","last_clean":"2023-07-06T20:14:01.239684+0000","last_became_active":"2023-07-06T20:14:00.064199+0000","last_became_peered":"2023-07-06T20:14:00.064199+0000","last_unstale":"2023-07-06T20:14:01.239684+0000","last_undegraded":"2023-07-06T20:14:01.239684+0000","last_fullsized":"2023-07-06T20:14:01.239684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047036+0000","last_change":"2023-07-06T20:14:00.056564+0000","last_active":"2023-07-06T20:14:01.047036+0000","last_peered":"2023-07-06T20:14:01.047036+0000","last_clean":"2023-07-06T20:14:01.047036+0000","last_became_active":"2023-07-06T20:14:00.056292+0000","last_became_peered":"2023-07-06T20:14:00.056292+0000","last_unstale":"2023-07-06T20:14:01.047036+0000","last_undegraded":"2023-07-06T20:14:01.047036+0000","last_fullsized":"2023-07-06T20:14:01.047036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239166+0000","last_change":"2023-07-06T20:14:00.068005+0000","last_active":"2023-07-06T20:14:01.239166+0000","last_peered":"2023-07-06T20:14:01.239166+0000","last_clean":"2023-07-06T20:14:01.239166+0000","last_became_active":"2023-07-06T20:14:00.067567+0000","last_became_peered":"2023-07-06T20:14:00.067567+0000","last_unstale":"2023-07-06T20:14:01.239166+0000","last_undegraded":"2023-07-06T20:14:01.239166+0000","last_fullsized":"2023-07-06T20:14:01.239166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239597+0000","last_change":"2023-07-06T20:14:00.057310+0000","last_active":"2023-07-06T20:14:01.239597+0000","last_peered":"2023-07-06T20:14:01.239597+0000","last_clean":"2023-07-06T20:14:01.239597+0000","last_became_active":"2023-07-06T20:14:00.057048+0000","last_became_peered":"2023-07-06T20:14:00.057048+0000","last_unstale":"2023-07-06T20:14:01.239597+0000","last_undegraded":"2023-07-06T20:14:01.239597+0000","last_fullsized":"2023-07-06T20:14:01.239597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239261+0000","last_change":"2023-07-06T20:14:00.053189+0000","last_active":"2023-07-06T20:14:01.239261+0000","last_peered":"2023-07-06T20:14:01.239261+0000","last_clean":"2023-07-06T20:14:01.239261+0000","last_became_active":"2023-07-06T20:14:00.052649+0000","last_became_peered":"2023-07-06T20:14:00.052649+0000","last_unstale":"2023-07-06T20:14:01.239261+0000","last_undegraded":"2023-07-06T20:14:01.239261+0000","last_fullsized":"2023-07-06T20:14:01.239261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:44:22.477453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240134+0000","last_change":"2023-07-06T20:14:00.056320+0000","last_active":"2023-07-06T20:14:01.240134+0000","last_peered":"2023-07-06T20:14:01.240134+0000","last_clean":"2023-07-06T20:14:01.240134+0000","last_became_active":"2023-07-06T20:14:00.055864+0000","last_became_peered":"2023-07-06T20:14:00.055864+0000","last_unstale":"2023-07-06T20:14:01.240134+0000","last_undegraded":"2023-07-06T20:14:01.240134+0000","last_fullsized":"2023-07-06T20:14:01.240134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047499+0000","last_change":"2023-07-06T20:14:00.063394+0000","last_active":"2023-07-06T20:14:01.047499+0000","last_peered":"2023-07-06T20:14:01.047499+0000","last_clean":"2023-07-06T20:14:01.047499+0000","last_became_active":"2023-07-06T20:14:00.063126+0000","last_became_peered":"2023-07-06T20:14:00.063126+0000","last_unstale":"2023-07-06T20:14:01.047499+0000","last_undegraded":"2023-07-06T20:14:01.047499+0000","last_fullsized":"2023-07-06T20:14:01.047499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239136+0000","last_change":"2023-07-06T20:14:00.063773+0000","last_active":"2023-07-06T20:14:01.239136+0000","last_peered":"2023-07-06T20:14:01.239136+0000","last_clean":"2023-07-06T20:14:01.239136+0000","last_became_active":"2023-07-06T20:14:00.063498+0000","last_became_peered":"2023-07-06T20:14:00.063498+0000","last_unstale":"2023-07-06T20:14:01.239136+0000","last_undegraded":"2023-07-06T20:14:01.239136+0000","last_fullsized":"2023-07-06T20:14:01.239136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:24.723303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239028+0000","last_change":"2023-07-06T20:14:00.037831+0000","last_active":"2023-07-06T20:14:01.239028+0000","last_peered":"2023-07-06T20:14:01.239028+0000","last_clean":"2023-07-06T20:14:01.239028+0000","last_became_active":"2023-07-06T20:14:00.037512+0000","last_became_peered":"2023-07-06T20:14:00.037512+0000","last_unstale":"2023-07-06T20:14:01.239028+0000","last_undegraded":"2023-07-06T20:14:01.239028+0000","last_fullsized":"2023-07-06T20:14:01.239028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:17.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240002+0000","last_change":"2023-07-06T20:14:00.049995+0000","last_active":"2023-07-06T20:14:01.240002+0000","last_peered":"2023-07-06T20:14:01.240002+0000","last_clean":"2023-07-06T20:14:01.240002+0000","last_became_active":"2023-07-06T20:14:00.049548+0000","last_became_peered":"2023-07-06T20:14:00.049548+0000","last_unstale":"2023-07-06T20:14:01.240002+0000","last_undegraded":"2023-07-06T20:14:01.240002+0000","last_fullsized":"2023-07-06T20:14:01.240002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239729+0000","last_change":"2023-07-06T20:14:00.067827+0000","last_active":"2023-07-06T20:14:01.239729+0000","last_peered":"2023-07-06T20:14:01.239729+0000","last_clean":"2023-07-06T20:14:01.239729+0000","last_became_active":"2023-07-06T20:14:00.067219+0000","last_became_peered":"2023-07-06T20:14:00.067219+0000","last_unstale":"2023-07-06T20:14:01.239729+0000","last_undegraded":"2023-07-06T20:14:01.239729+0000","last_fullsized":"2023-07-06T20:14:01.239729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047086+0000","last_change":"2023-07-06T20:14:00.056312+0000","last_active":"2023-07-06T20:14:01.047086+0000","last_peered":"2023-07-06T20:14:01.047086+0000","last_clean":"2023-07-06T20:14:01.047086+0000","last_became_active":"2023-07-06T20:14:00.055971+0000","last_became_peered":"2023-07-06T20:14:00.055971+0000","last_unstale":"2023-07-06T20:14:01.047086+0000","last_undegraded":"2023-07-06T20:14:01.047086+0000","last_fullsized":"2023-07-06T20:14:01.047086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239406+0000","last_change":"2023-07-06T20:14:00.052988+0000","last_active":"2023-07-06T20:14:01.239406+0000","last_peered":"2023-07-06T20:14:01.239406+0000","last_clean":"2023-07-06T20:14:01.239406+0000","last_became_active":"2023-07-06T20:14:00.052324+0000","last_became_peered":"2023-07-06T20:14:00.052324+0000","last_unstale":"2023-07-06T20:14:01.239406+0000","last_undegraded":"2023-07-06T20:14:01.239406+0000","last_fullsized":"2023-07-06T20:14:01.239406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:32.142546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047643+0000","last_change":"2023-07-06T20:14:00.054064+0000","last_active":"2023-07-06T20:14:01.047643+0000","last_peered":"2023-07-06T20:14:01.047643+0000","last_clean":"2023-07-06T20:14:01.047643+0000","last_became_active":"2023-07-06T20:14:00.053459+0000","last_became_peered":"2023-07-06T20:14:00.053459+0000","last_unstale":"2023-07-06T20:14:01.047643+0000","last_undegraded":"2023-07-06T20:14:01.047643+0000","last_fullsized":"2023-07-06T20:14:01.047643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239803+0000","last_change":"2023-07-06T20:14:00.047280+0000","last_active":"2023-07-06T20:14:01.239803+0000","last_peered":"2023-07-06T20:14:01.239803+0000","last_clean":"2023-07-06T20:14:01.239803+0000","last_became_active":"2023-07-06T20:14:00.046943+0000","last_became_peered":"2023-07-06T20:14:00.046943+0000","last_unstale":"2023-07-06T20:14:01.239803+0000","last_undegraded":"2023-07-06T20:14:01.239803+0000","last_fullsized":"2023-07-06T20:14:01.239803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047192+0000","last_change":"2023-07-06T20:14:00.054317+0000","last_active":"2023-07-06T20:14:01.047192+0000","last_peered":"2023-07-06T20:14:01.047192+0000","last_clean":"2023-07-06T20:14:01.047192+0000","last_became_active":"2023-07-06T20:14:00.053672+0000","last_became_peered":"2023-07-06T20:14:00.053672+0000","last_unstale":"2023-07-06T20:14:01.047192+0000","last_undegraded":"2023-07-06T20:14:01.047192+0000","last_fullsized":"2023-07-06T20:14:01.047192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240189+0000","last_change":"2023-07-06T20:14:00.052959+0000","last_active":"2023-07-06T20:14:01.240189+0000","last_peered":"2023-07-06T20:14:01.240189+0000","last_clean":"2023-07-06T20:14:01.240189+0000","last_became_active":"2023-07-06T20:14:00.052605+0000","last_became_peered":"2023-07-06T20:14:00.052605+0000","last_unstale":"2023-07-06T20:14:01.240189+0000","last_undegraded":"2023-07-06T20:14:01.240189+0000","last_fullsized":"2023-07-06T20:14:01.240189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239515+0000","last_change":"2023-07-06T20:14:00.065147+0000","last_active":"2023-07-06T20:14:01.239515+0000","last_peered":"2023-07-06T20:14:01.239515+0000","last_clean":"2023-07-06T20:14:01.239515+0000","last_became_active":"2023-07-06T20:14:00.064652+0000","last_became_peered":"2023-07-06T20:14:00.064652+0000","last_unstale":"2023-07-06T20:14:01.239515+0000","last_undegraded":"2023-07-06T20:14:01.239515+0000","last_fullsized":"2023-07-06T20:14:01.239515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046578+0000","last_change":"2023-07-06T20:14:00.054578+0000","last_active":"2023-07-06T20:14:01.046578+0000","last_peered":"2023-07-06T20:14:01.046578+0000","last_clean":"2023-07-06T20:14:01.046578+0000","last_became_active":"2023-07-06T20:14:00.053689+0000","last_became_peered":"2023-07-06T20:14:00.053689+0000","last_unstale":"2023-07-06T20:14:01.046578+0000","last_undegraded":"2023-07-06T20:14:01.046578+0000","last_fullsized":"2023-07-06T20:14:01.046578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239287+0000","last_change":"2023-07-06T20:14:00.067375+0000","last_active":"2023-07-06T20:14:01.239287+0000","last_peered":"2023-07-06T20:14:01.239287+0000","last_clean":"2023-07-06T20:14:01.239287+0000","last_became_active":"2023-07-06T20:14:00.066909+0000","last_became_peered":"2023-07-06T20:14:00.066909+0000","last_unstale":"2023-07-06T20:14:01.239287+0000","last_undegraded":"2023-07-06T20:14:01.239287+0000","last_fullsized":"2023-07-06T20:14:01.239287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240385+0000","last_change":"2023-07-06T20:14:00.064820+0000","last_active":"2023-07-06T20:14:01.240385+0000","last_peered":"2023-07-06T20:14:01.240385+0000","last_clean":"2023-07-06T20:14:01.240385+0000","last_became_active":"2023-07-06T20:14:00.064585+0000","last_became_peered":"2023-07-06T20:14:00.064585+0000","last_unstale":"2023-07-06T20:14:01.240385+0000","last_undegraded":"2023-07-06T20:14:01.240385+0000","last_fullsized":"2023-07-06T20:14:01.240385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240518+0000","last_change":"2023-07-06T20:14:00.055061+0000","last_active":"2023-07-06T20:14:01.240518+0000","last_peered":"2023-07-06T20:14:01.240518+0000","last_clean":"2023-07-06T20:14:01.240518+0000","last_became_active":"2023-07-06T20:14:00.054795+0000","last_became_peered":"2023-07-06T20:14:00.054795+0000","last_unstale":"2023-07-06T20:14:01.240518+0000","last_undegraded":"2023-07-06T20:14:01.240518+0000","last_fullsized":"2023-07-06T20:14:01.240518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240667+0000","last_change":"2023-07-06T20:14:00.057631+0000","last_active":"2023-07-06T20:14:01.240667+0000","last_peered":"2023-07-06T20:14:01.240667+0000","last_clean":"2023-07-06T20:14:01.240667+0000","last_became_active":"2023-07-06T20:14:00.056582+0000","last_became_peered":"2023-07-06T20:14:00.056582+0000","last_unstale":"2023-07-06T20:14:01.240667+0000","last_undegraded":"2023-07-06T20:14:01.240667+0000","last_fullsized":"2023-07-06T20:14:01.240667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047284+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:14:01.047284+0000","last_peered":"2023-07-06T20:14:01.047284+0000","last_clean":"2023-07-06T20:14:01.047284+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:14:01.047284+0000","last_undegraded":"2023-07-06T20:14:01.047284+0000","last_fullsized":"2023-07-06T20:14:01.047284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047084+0000","last_change":"2023-07-06T20:14:00.067982+0000","last_active":"2023-07-06T20:14:01.047084+0000","last_peered":"2023-07-06T20:14:01.047084+0000","last_clean":"2023-07-06T20:14:01.047084+0000","last_became_active":"2023-07-06T20:14:00.067690+0000","last_became_peered":"2023-07-06T20:14:00.067690+0000","last_unstale":"2023-07-06T20:14:01.047084+0000","last_undegraded":"2023-07-06T20:14:01.047084+0000","last_fullsized":"2023-07-06T20:14:01.047084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239754+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:14:01.239754+0000","last_peered":"2023-07-06T20:14:01.239754+0000","last_clean":"2023-07-06T20:14:01.239754+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:14:01.239754+0000","last_undegraded":"2023-07-06T20:14:01.239754+0000","last_fullsized":"2023-07-06T20:14:01.239754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240204+0000","last_change":"2023-07-06T20:14:00.061405+0000","last_active":"2023-07-06T20:14:01.240204+0000","last_peered":"2023-07-06T20:14:01.240204+0000","last_clean":"2023-07-06T20:14:01.240204+0000","last_became_active":"2023-07-06T20:14:00.060981+0000","last_became_peered":"2023-07-06T20:14:00.060981+0000","last_unstale":"2023-07-06T20:14:01.240204+0000","last_undegraded":"2023-07-06T20:14:01.240204+0000","last_fullsized":"2023-07-06T20:14:01.240204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047136+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:14:01.047136+0000","last_peered":"2023-07-06T20:14:01.047136+0000","last_clean":"2023-07-06T20:14:01.047136+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:14:01.047136+0000","last_undegraded":"2023-07-06T20:14:01.047136+0000","last_fullsized":"2023-07-06T20:14:01.047136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239984+0000","last_change":"2023-07-06T20:14:00.052443+0000","last_active":"2023-07-06T20:14:01.239984+0000","last_peered":"2023-07-06T20:14:01.239984+0000","last_clean":"2023-07-06T20:14:01.239984+0000","last_became_active":"2023-07-06T20:14:00.052132+0000","last_became_peered":"2023-07-06T20:14:00.052132+0000","last_unstale":"2023-07-06T20:14:01.239984+0000","last_undegraded":"2023-07-06T20:14:01.239984+0000","last_fullsized":"2023-07-06T20:14:01.239984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:53.471708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239579+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:14:01.239579+0000","last_peered":"2023-07-06T20:14:01.239579+0000","last_clean":"2023-07-06T20:14:01.239579+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:14:01.239579+0000","last_undegraded":"2023-07-06T20:14:01.239579+0000","last_fullsized":"2023-07-06T20:14:01.239579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:14:00.067811+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:14:00.067528+0000","last_became_peered":"2023-07-06T20:14:00.067528+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239990+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:14:01.239990+0000","last_peered":"2023-07-06T20:14:01.239990+0000","last_clean":"2023-07-06T20:14:01.239990+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:14:01.239990+0000","last_undegraded":"2023-07-06T20:14:01.239990+0000","last_fullsized":"2023-07-06T20:14:01.239990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046539+0000","last_change":"2023-07-06T20:14:00.060824+0000","last_active":"2023-07-06T20:14:01.046539+0000","last_peered":"2023-07-06T20:14:01.046539+0000","last_clean":"2023-07-06T20:14:01.046539+0000","last_became_active":"2023-07-06T20:14:00.059809+0000","last_became_peered":"2023-07-06T20:14:00.059809+0000","last_unstale":"2023-07-06T20:14:01.046539+0000","last_undegraded":"2023-07-06T20:14:01.046539+0000","last_fullsized":"2023-07-06T20:14:01.046539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240120+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:14:01.240120+0000","last_peered":"2023-07-06T20:14:01.240120+0000","last_clean":"2023-07-06T20:14:01.240120+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.240120+0000","last_undegraded":"2023-07-06T20:14:01.240120+0000","last_fullsized":"2023-07-06T20:14:01.240120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:56.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240650+0000","last_change":"2023-07-06T20:14:00.056152+0000","last_active":"2023-07-06T20:14:01.240650+0000","last_peered":"2023-07-06T20:14:01.240650+0000","last_clean":"2023-07-06T20:14:01.240650+0000","last_became_active":"2023-07-06T20:14:00.055844+0000","last_became_peered":"2023-07-06T20:14:00.055844+0000","last_unstale":"2023-07-06T20:14:01.240650+0000","last_undegraded":"2023-07-06T20:14:01.240650+0000","last_fullsized":"2023-07-06T20:14:01.240650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047386+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:14:01.047386+0000","last_peered":"2023-07-06T20:14:01.047386+0000","last_clean":"2023-07-06T20:14:01.047386+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:14:01.047386+0000","last_undegraded":"2023-07-06T20:14:01.047386+0000","last_fullsized":"2023-07-06T20:14:01.047386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047231+0000","last_change":"2023-07-06T20:14:00.054939+0000","last_active":"2023-07-06T20:14:01.047231+0000","last_peered":"2023-07-06T20:14:01.047231+0000","last_clean":"2023-07-06T20:14:01.047231+0000","last_became_active":"2023-07-06T20:14:00.054354+0000","last_became_peered":"2023-07-06T20:14:00.054354+0000","last_unstale":"2023-07-06T20:14:01.047231+0000","last_undegraded":"2023-07-06T20:14:01.047231+0000","last_fullsized":"2023-07-06T20:14:01.047231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240033+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:14:01.240033+0000","last_peered":"2023-07-06T20:14:01.240033+0000","last_clean":"2023-07-06T20:14:01.240033+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:14:01.240033+0000","last_undegraded":"2023-07-06T20:14:01.240033+0000","last_fullsized":"2023-07-06T20:14:01.240033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:26.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046831+0000","last_change":"2023-07-06T20:14:00.056179+0000","last_active":"2023-07-06T20:14:01.046831+0000","last_peered":"2023-07-06T20:14:01.046831+0000","last_clean":"2023-07-06T20:14:01.046831+0000","last_became_active":"2023-07-06T20:14:00.055858+0000","last_became_peered":"2023-07-06T20:14:00.055858+0000","last_unstale":"2023-07-06T20:14:01.046831+0000","last_undegraded":"2023-07-06T20:14:01.046831+0000","last_fullsized":"2023-07-06T20:14:01.046831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239368+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:14:01.239368+0000","last_peered":"2023-07-06T20:14:01.239368+0000","last_clean":"2023-07-06T20:14:01.239368+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:14:01.239368+0000","last_undegraded":"2023-07-06T20:14:01.239368+0000","last_fullsized":"2023-07-06T20:14:01.239368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:48.108641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240807+0000","last_change":"2023-07-06T20:14:00.056298+0000","last_active":"2023-07-06T20:14:01.240807+0000","last_peered":"2023-07-06T20:14:01.240807+0000","last_clean":"2023-07-06T20:14:01.240807+0000","last_became_active":"2023-07-06T20:14:00.055932+0000","last_became_peered":"2023-07-06T20:14:00.055932+0000","last_unstale":"2023-07-06T20:14:01.240807+0000","last_undegraded":"2023-07-06T20:14:01.240807+0000","last_fullsized":"2023-07-06T20:14:01.240807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239561+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:14:01.239561+0000","last_peered":"2023-07-06T20:14:01.239561+0000","last_clean":"2023-07-06T20:14:01.239561+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:14:01.239561+0000","last_undegraded":"2023-07-06T20:14:01.239561+0000","last_fullsized":"2023-07-06T20:14:01.239561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240113+0000","last_change":"2023-07-06T20:14:00.064964+0000","last_active":"2023-07-06T20:14:01.240113+0000","last_peered":"2023-07-06T20:14:01.240113+0000","last_clean":"2023-07-06T20:14:01.240113+0000","last_became_active":"2023-07-06T20:14:00.064266+0000","last_became_peered":"2023-07-06T20:14:00.064266+0000","last_unstale":"2023-07-06T20:14:01.240113+0000","last_undegraded":"2023-07-06T20:14:01.240113+0000","last_fullsized":"2023-07-06T20:14:01.240113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239123+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:14:01.239123+0000","last_peered":"2023-07-06T20:14:01.239123+0000","last_clean":"2023-07-06T20:14:01.239123+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:14:01.239123+0000","last_undegraded":"2023-07-06T20:14:01.239123+0000","last_fullsized":"2023-07-06T20:14:01.239123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047315+0000","last_change":"2023-07-06T20:14:00.061173+0000","last_active":"2023-07-06T20:14:01.047315+0000","last_peered":"2023-07-06T20:14:01.047315+0000","last_clean":"2023-07-06T20:14:01.047315+0000","last_became_active":"2023-07-06T20:14:00.060346+0000","last_became_peered":"2023-07-06T20:14:00.060346+0000","last_unstale":"2023-07-06T20:14:01.047315+0000","last_undegraded":"2023-07-06T20:14:01.047315+0000","last_fullsized":"2023-07-06T20:14:01.047315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239999+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:14:01.239999+0000","last_peered":"2023-07-06T20:14:01.239999+0000","last_clean":"2023-07-06T20:14:01.239999+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:14:01.239999+0000","last_undegraded":"2023-07-06T20:14:01.239999+0000","last_fullsized":"2023-07-06T20:14:01.239999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:37.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239775+0000","last_change":"2023-07-06T20:14:00.061044+0000","last_active":"2023-07-06T20:14:01.239775+0000","last_peered":"2023-07-06T20:14:01.239775+0000","last_clean":"2023-07-06T20:14:01.239775+0000","last_became_active":"2023-07-06T20:14:00.060737+0000","last_became_peered":"2023-07-06T20:14:00.060737+0000","last_unstale":"2023-07-06T20:14:01.239775+0000","last_undegraded":"2023-07-06T20:14:01.239775+0000","last_fullsized":"2023-07-06T20:14:01.239775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240322+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:14:01.240322+0000","last_peered":"2023-07-06T20:14:01.240322+0000","last_clean":"2023-07-06T20:14:01.240322+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:14:01.240322+0000","last_undegraded":"2023-07-06T20:14:01.240322+0000","last_fullsized":"2023-07-06T20:14:01.240322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.062318+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.062063+0000","last_became_peered":"2023-07-06T20:14:00.062063+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240374+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:14:01.240374+0000","last_peered":"2023-07-06T20:14:01.240374+0000","last_clean":"2023-07-06T20:14:01.240374+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:14:01.240374+0000","last_undegraded":"2023-07-06T20:14:01.240374+0000","last_fullsized":"2023-07-06T20:14:01.240374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:11.129495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240536+0000","last_change":"2023-07-06T20:14:00.064988+0000","last_active":"2023-07-06T20:14:01.240536+0000","last_peered":"2023-07-06T20:14:01.240536+0000","last_clean":"2023-07-06T20:14:01.240536+0000","last_became_active":"2023-07-06T20:14:00.064581+0000","last_became_peered":"2023-07-06T20:14:00.064581+0000","last_unstale":"2023-07-06T20:14:01.240536+0000","last_undegraded":"2023-07-06T20:14:01.240536+0000","last_fullsized":"2023-07-06T20:14:01.240536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:10.399206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239427+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:14:01.239427+0000","last_peered":"2023-07-06T20:14:01.239427+0000","last_clean":"2023-07-06T20:14:01.239427+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:14:01.239427+0000","last_undegraded":"2023-07-06T20:14:01.239427+0000","last_fullsized":"2023-07-06T20:14:01.239427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:18.117668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239302+0000","last_change":"2023-07-06T20:14:00.065729+0000","last_active":"2023-07-06T20:14:01.239302+0000","last_peered":"2023-07-06T20:14:01.239302+0000","last_clean":"2023-07-06T20:14:01.239302+0000","last_became_active":"2023-07-06T20:14:00.064827+0000","last_became_peered":"2023-07-06T20:14:00.064827+0000","last_unstale":"2023-07-06T20:14:01.239302+0000","last_undegraded":"2023-07-06T20:14:01.239302+0000","last_fullsized":"2023-07-06T20:14:01.239302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:21:48.022430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046583+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:14:01.046583+0000","last_peered":"2023-07-06T20:14:01.046583+0000","last_clean":"2023-07-06T20:14:01.046583+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.046583+0000","last_undegraded":"2023-07-06T20:14:01.046583+0000","last_fullsized":"2023-07-06T20:14:01.046583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:18.188230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047260+0000","last_change":"2023-07-06T20:14:00.054542+0000","last_active":"2023-07-06T20:14:01.047260+0000","last_peered":"2023-07-06T20:14:01.047260+0000","last_clean":"2023-07-06T20:14:01.047260+0000","last_became_active":"2023-07-06T20:14:00.053987+0000","last_became_peered":"2023-07-06T20:14:00.053987+0000","last_unstale":"2023-07-06T20:14:01.047260+0000","last_undegraded":"2023-07-06T20:14:01.047260+0000","last_fullsized":"2023-07-06T20:14:01.047260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240427+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:14:01.240427+0000","last_peered":"2023-07-06T20:14:01.240427+0000","last_clean":"2023-07-06T20:14:01.240427+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:14:01.240427+0000","last_undegraded":"2023-07-06T20:14:01.240427+0000","last_fullsized":"2023-07-06T20:14:01.240427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:32.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047708+0000","last_change":"2023-07-06T20:14:00.052728+0000","last_active":"2023-07-06T20:14:01.047708+0000","last_peered":"2023-07-06T20:14:01.047708+0000","last_clean":"2023-07-06T20:14:01.047708+0000","last_became_active":"2023-07-06T20:14:00.052437+0000","last_became_peered":"2023-07-06T20:14:00.052437+0000","last_unstale":"2023-07-06T20:14:01.047708+0000","last_undegraded":"2023-07-06T20:14:01.047708+0000","last_fullsized":"2023-07-06T20:14:01.047708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:27.783430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047897+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:14:01.047897+0000","last_peered":"2023-07-06T20:14:01.047897+0000","last_clean":"2023-07-06T20:14:01.047897+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:14:01.047897+0000","last_undegraded":"2023-07-06T20:14:01.047897+0000","last_fullsized":"2023-07-06T20:14:01.047897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.053789+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.053148+0000","last_became_peered":"2023-07-06T20:14:00.053148+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:57.270521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046721+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:14:01.046721+0000","last_peered":"2023-07-06T20:14:01.046721+0000","last_clean":"2023-07-06T20:14:01.046721+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:14:01.046721+0000","last_undegraded":"2023-07-06T20:14:01.046721+0000","last_fullsized":"2023-07-06T20:14:01.046721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240548+0000","last_change":"2023-07-06T20:14:00.056310+0000","last_active":"2023-07-06T20:14:01.240548+0000","last_peered":"2023-07-06T20:14:01.240548+0000","last_clean":"2023-07-06T20:14:01.240548+0000","last_became_active":"2023-07-06T20:14:00.055916+0000","last_became_peered":"2023-07-06T20:14:00.055916+0000","last_unstale":"2023-07-06T20:14:01.240548+0000","last_undegraded":"2023-07-06T20:14:01.240548+0000","last_fullsized":"2023-07-06T20:14:01.240548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239879+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:14:01.239879+0000","last_peered":"2023-07-06T20:14:01.239879+0000","last_clean":"2023-07-06T20:14:01.239879+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:14:01.239879+0000","last_undegraded":"2023-07-06T20:14:01.239879+0000","last_fullsized":"2023-07-06T20:14:01.239879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:12.079947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240842+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:14:01.240842+0000","last_peered":"2023-07-06T20:14:01.240842+0000","last_clean":"2023-07-06T20:14:01.240842+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:14:01.240842+0000","last_undegraded":"2023-07-06T20:14:01.240842+0000","last_fullsized":"2023-07-06T20:14:01.240842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240741+0000","last_change":"2023-07-06T20:14:00.055724+0000","last_active":"2023-07-06T20:14:01.240741+0000","last_peered":"2023-07-06T20:14:01.240741+0000","last_clean":"2023-07-06T20:14:01.240741+0000","last_became_active":"2023-07-06T20:14:00.055440+0000","last_became_peered":"2023-07-06T20:14:00.055440+0000","last_unstale":"2023-07-06T20:14:01.240741+0000","last_undegraded":"2023-07-06T20:14:01.240741+0000","last_fullsized":"2023-07-06T20:14:01.240741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240130+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:14:01.240130+0000","last_peered":"2023-07-06T20:14:01.240130+0000","last_clean":"2023-07-06T20:14:01.240130+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:14:01.240130+0000","last_undegraded":"2023-07-06T20:14:01.240130+0000","last_fullsized":"2023-07-06T20:14:01.240130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:50.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047412+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:14:01.047412+0000","last_peered":"2023-07-06T20:14:01.047412+0000","last_clean":"2023-07-06T20:14:01.047412+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:14:01.047412+0000","last_undegraded":"2023-07-06T20:14:01.047412+0000","last_fullsized":"2023-07-06T20:14:01.047412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:17.962306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240019+0000","last_change":"2023-07-06T20:14:00.055529+0000","last_active":"2023-07-06T20:14:01.240019+0000","last_peered":"2023-07-06T20:14:01.240019+0000","last_clean":"2023-07-06T20:14:01.240019+0000","last_became_active":"2023-07-06T20:14:00.054757+0000","last_became_peered":"2023-07-06T20:14:00.054757+0000","last_unstale":"2023-07-06T20:14:01.240019+0000","last_undegraded":"2023-07-06T20:14:01.240019+0000","last_fullsized":"2023-07-06T20:14:01.240019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238739+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:14:01.238739+0000","last_peered":"2023-07-06T20:14:01.238739+0000","last_clean":"2023-07-06T20:14:01.238739+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:14:01.238739+0000","last_undegraded":"2023-07-06T20:14:01.238739+0000","last_fullsized":"2023-07-06T20:14:01.238739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239367+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:14:01.239367+0000","last_peered":"2023-07-06T20:14:01.239367+0000","last_clean":"2023-07-06T20:14:01.239367+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:14:01.239367+0000","last_undegraded":"2023-07-06T20:14:01.239367+0000","last_fullsized":"2023-07-06T20:14:01.239367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240403+0000","last_change":"2023-07-06T20:14:00.062155+0000","last_active":"2023-07-06T20:14:01.240403+0000","last_peered":"2023-07-06T20:14:01.240403+0000","last_clean":"2023-07-06T20:14:01.240403+0000","last_became_active":"2023-07-06T20:14:00.061848+0000","last_became_peered":"2023-07-06T20:14:00.061848+0000","last_unstale":"2023-07-06T20:14:01.240403+0000","last_undegraded":"2023-07-06T20:14:01.240403+0000","last_fullsized":"2023-07-06T20:14:01.240403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239665+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:14:01.239665+0000","last_peered":"2023-07-06T20:14:01.239665+0000","last_clean":"2023-07-06T20:14:01.239665+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:14:01.239665+0000","last_undegraded":"2023-07-06T20:14:01.239665+0000","last_fullsized":"2023-07-06T20:14:01.239665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:53.676445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239234+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:14:01.239234+0000","last_peered":"2023-07-06T20:14:01.239234+0000","last_clean":"2023-07-06T20:14:01.239234+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:14:01.239234+0000","last_undegraded":"2023-07-06T20:14:01.239234+0000","last_fullsized":"2023-07-06T20:14:01.239234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:04.461531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046254+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:14:01.046254+0000","last_peered":"2023-07-06T20:14:01.046254+0000","last_clean":"2023-07-06T20:14:01.046254+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:14:01.046254+0000","last_undegraded":"2023-07-06T20:14:01.046254+0000","last_fullsized":"2023-07-06T20:14:01.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239372+0000","last_change":"2023-07-06T20:14:00.062536+0000","last_active":"2023-07-06T20:14:01.239372+0000","last_peered":"2023-07-06T20:14:01.239372+0000","last_clean":"2023-07-06T20:14:01.239372+0000","last_became_active":"2023-07-06T20:14:00.062189+0000","last_became_peered":"2023-07-06T20:14:00.062189+0000","last_unstale":"2023-07-06T20:14:01.239372+0000","last_undegraded":"2023-07-06T20:14:01.239372+0000","last_fullsized":"2023-07-06T20:14:01.239372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:07.175614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239383+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:14:01.239383+0000","last_peered":"2023-07-06T20:14:01.239383+0000","last_clean":"2023-07-06T20:14:01.239383+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:14:01.239383+0000","last_undegraded":"2023-07-06T20:14:01.239383+0000","last_fullsized":"2023-07-06T20:14:01.239383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:38.150080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239126+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:14:01.239126+0000","last_peered":"2023-07-06T20:14:01.239126+0000","last_clean":"2023-07-06T20:14:01.239126+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:14:01.239126+0000","last_undegraded":"2023-07-06T20:14:01.239126+0000","last_fullsized":"2023-07-06T20:14:01.239126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:42.107407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240485+0000","last_change":"2023-07-06T20:14:00.065377+0000","last_active":"2023-07-06T20:14:01.240485+0000","last_peered":"2023-07-06T20:14:01.240485+0000","last_clean":"2023-07-06T20:14:01.240485+0000","last_became_active":"2023-07-06T20:14:00.064627+0000","last_became_peered":"2023-07-06T20:14:00.064627+0000","last_unstale":"2023-07-06T20:14:01.240485+0000","last_undegraded":"2023-07-06T20:14:01.240485+0000","last_fullsized":"2023-07-06T20:14:01.240485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240293+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:14:01.240293+0000","last_peered":"2023-07-06T20:14:01.240293+0000","last_clean":"2023-07-06T20:14:01.240293+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:14:01.240293+0000","last_undegraded":"2023-07-06T20:14:01.240293+0000","last_fullsized":"2023-07-06T20:14:01.240293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:41:59.625038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240045+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:14:01.240045+0000","last_peered":"2023-07-06T20:14:01.240045+0000","last_clean":"2023-07-06T20:14:01.240045+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:14:01.240045+0000","last_undegraded":"2023-07-06T20:14:01.240045+0000","last_fullsized":"2023-07-06T20:14:01.240045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239757+0000","last_change":"2023-07-06T20:14:00.057341+0000","last_active":"2023-07-06T20:14:01.239757+0000","last_peered":"2023-07-06T20:14:01.239757+0000","last_clean":"2023-07-06T20:14:01.239757+0000","last_became_active":"2023-07-06T20:14:00.057077+0000","last_became_peered":"2023-07-06T20:14:00.057077+0000","last_unstale":"2023-07-06T20:14:01.239757+0000","last_undegraded":"2023-07-06T20:14:01.239757+0000","last_fullsized":"2023-07-06T20:14:01.239757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047029+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:14:01.047029+0000","last_peered":"2023-07-06T20:14:01.047029+0000","last_clean":"2023-07-06T20:14:01.047029+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:14:01.047029+0000","last_undegraded":"2023-07-06T20:14:01.047029+0000","last_fullsized":"2023-07-06T20:14:01.047029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:37.193187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240442+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:14:01.240442+0000","last_peered":"2023-07-06T20:14:01.240442+0000","last_clean":"2023-07-06T20:14:01.240442+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:14:01.240442+0000","last_undegraded":"2023-07-06T20:14:01.240442+0000","last_fullsized":"2023-07-06T20:14:01.240442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:03.076475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046144+0000","last_change":"2023-07-06T20:14:00.066131+0000","last_active":"2023-07-06T20:14:01.046144+0000","last_peered":"2023-07-06T20:14:01.046144+0000","last_clean":"2023-07-06T20:14:01.046144+0000","last_became_active":"2023-07-06T20:14:00.065793+0000","last_became_peered":"2023-07-06T20:14:00.065793+0000","last_unstale":"2023-07-06T20:14:01.046144+0000","last_undegraded":"2023-07-06T20:14:01.046144+0000","last_fullsized":"2023-07-06T20:14:01.046144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:49.111646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239813+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:14:01.239813+0000","last_peered":"2023-07-06T20:14:01.239813+0000","last_clean":"2023-07-06T20:14:01.239813+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:14:01.239813+0000","last_undegraded":"2023-07-06T20:14:01.239813+0000","last_fullsized":"2023-07-06T20:14:01.239813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:17.841824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240384+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:14:01.240384+0000","last_peered":"2023-07-06T20:14:01.240384+0000","last_clean":"2023-07-06T20:14:01.240384+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:14:01.240384+0000","last_undegraded":"2023-07-06T20:14:01.240384+0000","last_fullsized":"2023-07-06T20:14:01.240384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047806+0000","last_change":"2023-07-06T20:14:00.055463+0000","last_active":"2023-07-06T20:14:01.047806+0000","last_peered":"2023-07-06T20:14:01.047806+0000","last_clean":"2023-07-06T20:14:01.047806+0000","last_became_active":"2023-07-06T20:14:00.055196+0000","last_became_peered":"2023-07-06T20:14:00.055196+0000","last_unstale":"2023-07-06T20:14:01.047806+0000","last_undegraded":"2023-07-06T20:14:01.047806+0000","last_fullsized":"2023-07-06T20:14:01.047806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:42:06.665764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239438+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:14:01.239438+0000","last_peered":"2023-07-06T20:14:01.239438+0000","last_clean":"2023-07-06T20:14:01.239438+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:14:01.239438+0000","last_undegraded":"2023-07-06T20:14:01.239438+0000","last_fullsized":"2023-07-06T20:14:01.239438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239401+0000","last_change":"2023-07-06T20:14:00.067006+0000","last_active":"2023-07-06T20:14:01.239401+0000","last_peered":"2023-07-06T20:14:01.239401+0000","last_clean":"2023-07-06T20:14:01.239401+0000","last_became_active":"2023-07-06T20:14:00.066735+0000","last_became_peered":"2023-07-06T20:14:00.066735+0000","last_unstale":"2023-07-06T20:14:01.239401+0000","last_undegraded":"2023-07-06T20:14:01.239401+0000","last_fullsized":"2023-07-06T20:14:01.239401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:37.878199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239281+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:14:01.239281+0000","last_peered":"2023-07-06T20:14:01.239281+0000","last_clean":"2023-07-06T20:14:01.239281+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:14:01.239281+0000","last_undegraded":"2023-07-06T20:14:01.239281+0000","last_fullsized":"2023-07-06T20:14:01.239281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047642+0000","last_change":"2023-07-06T20:14:00.054948+0000","last_active":"2023-07-06T20:14:01.047642+0000","last_peered":"2023-07-06T20:14:01.047642+0000","last_clean":"2023-07-06T20:14:01.047642+0000","last_became_active":"2023-07-06T20:14:00.054677+0000","last_became_peered":"2023-07-06T20:14:00.054677+0000","last_unstale":"2023-07-06T20:14:01.047642+0000","last_undegraded":"2023-07-06T20:14:01.047642+0000","last_fullsized":"2023-07-06T20:14:01.047642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:38.225314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240958+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:14:01.240958+0000","last_peered":"2023-07-06T20:14:01.240958+0000","last_clean":"2023-07-06T20:14:01.240958+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:14:01.240958+0000","last_undegraded":"2023-07-06T20:14:01.240958+0000","last_fullsized":"2023-07-06T20:14:01.240958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:37.736933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047478+0000","last_change":"2023-07-06T20:14:00.053372+0000","last_active":"2023-07-06T20:14:01.047478+0000","last_peered":"2023-07-06T20:14:01.047478+0000","last_clean":"2023-07-06T20:14:01.047478+0000","last_became_active":"2023-07-06T20:14:00.053043+0000","last_became_peered":"2023-07-06T20:14:00.053043+0000","last_unstale":"2023-07-06T20:14:01.047478+0000","last_undegraded":"2023-07-06T20:14:01.047478+0000","last_fullsized":"2023-07-06T20:14:01.047478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046413+0000","last_change":"2023-07-06T20:14:00.038980+0000","last_active":"2023-07-06T20:14:01.046413+0000","last_peered":"2023-07-06T20:14:01.046413+0000","last_clean":"2023-07-06T20:14:01.046413+0000","last_became_active":"2023-07-06T20:14:00.038706+0000","last_became_peered":"2023-07-06T20:14:00.038706+0000","last_unstale":"2023-07-06T20:14:01.046413+0000","last_undegraded":"2023-07-06T20:14:01.046413+0000","last_fullsized":"2023-07-06T20:14:01.046413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239873+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:14:01.239873+0000","last_peered":"2023-07-06T20:14:01.239873+0000","last_clean":"2023-07-06T20:14:01.239873+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:14:01.239873+0000","last_undegraded":"2023-07-06T20:14:01.239873+0000","last_fullsized":"2023-07-06T20:14:01.239873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047021+0000","last_change":"2023-07-06T20:14:00.049540+0000","last_active":"2023-07-06T20:14:01.047021+0000","last_peered":"2023-07-06T20:14:01.047021+0000","last_clean":"2023-07-06T20:14:01.047021+0000","last_became_active":"2023-07-06T20:14:00.049260+0000","last_became_peered":"2023-07-06T20:14:00.049260+0000","last_unstale":"2023-07-06T20:14:01.047021+0000","last_undegraded":"2023-07-06T20:14:01.047021+0000","last_fullsized":"2023-07-06T20:14:01.047021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239288+0000","last_change":"2023-07-06T20:14:00.048736+0000","last_active":"2023-07-06T20:14:01.239288+0000","last_peered":"2023-07-06T20:14:01.239288+0000","last_clean":"2023-07-06T20:14:01.239288+0000","last_became_active":"2023-07-06T20:14:00.048418+0000","last_became_peered":"2023-07-06T20:14:00.048418+0000","last_unstale":"2023-07-06T20:14:01.239288+0000","last_undegraded":"2023-07-06T20:14:01.239288+0000","last_fullsized":"2023-07-06T20:14:01.239288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046375+0000","last_change":"2023-07-06T20:14:00.067909+0000","last_active":"2023-07-06T20:14:01.046375+0000","last_peered":"2023-07-06T20:14:01.046375+0000","last_clean":"2023-07-06T20:14:01.046375+0000","last_became_active":"2023-07-06T20:14:00.067600+0000","last_became_peered":"2023-07-06T20:14:00.067600+0000","last_unstale":"2023-07-06T20:14:01.046375+0000","last_undegraded":"2023-07-06T20:14:01.046375+0000","last_fullsized":"2023-07-06T20:14:01.046375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047258+0000","last_change":"2023-07-06T20:14:00.058103+0000","last_active":"2023-07-06T20:14:01.047258+0000","last_peered":"2023-07-06T20:14:01.047258+0000","last_clean":"2023-07-06T20:14:01.047258+0000","last_became_active":"2023-07-06T20:14:00.057800+0000","last_became_peered":"2023-07-06T20:14:00.057800+0000","last_unstale":"2023-07-06T20:14:01.047258+0000","last_undegraded":"2023-07-06T20:14:01.047258+0000","last_fullsized":"2023-07-06T20:14:01.047258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046528+0000","last_change":"2023-07-06T20:14:00.061079+0000","last_active":"2023-07-06T20:14:01.046528+0000","last_peered":"2023-07-06T20:14:01.046528+0000","last_clean":"2023-07-06T20:14:01.046528+0000","last_became_active":"2023-07-06T20:14:00.059768+0000","last_became_peered":"2023-07-06T20:14:00.059768+0000","last_unstale":"2023-07-06T20:14:01.046528+0000","last_undegraded":"2023-07-06T20:14:01.046528+0000","last_fullsized":"2023-07-06T20:14:01.046528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239855+0000","last_change":"2023-07-06T20:14:00.056153+0000","last_active":"2023-07-06T20:14:01.239855+0000","last_peered":"2023-07-06T20:14:01.239855+0000","last_clean":"2023-07-06T20:14:01.239855+0000","last_became_active":"2023-07-06T20:14:00.055830+0000","last_became_peered":"2023-07-06T20:14:00.055830+0000","last_unstale":"2023-07-06T20:14:01.239855+0000","last_undegraded":"2023-07-06T20:14:01.239855+0000","last_fullsized":"2023-07-06T20:14:01.239855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:09.968 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T20:14:09.969 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-06T20:14:09.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-06T20:14:09.969 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-06T20:14:09.970 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-06T20:14:10.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.204+0000 7fb78d1ff700 1 Processor -- start 2023-07-06T20:14:10.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.205+0000 7fb78d1ff700 1 -- start start 2023-07-06T20:14:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78d1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb78815fca0 0x7fb7880d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78d1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78d1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7880b60e0 0x7fb7880d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78d1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7880cf780 con 0x7fb7880b60e0 2023-07-06T20:14:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78d1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7880cd490 con 0x7fb78815fca0 2023-07-06T20:14:10.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78d1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880cd770 con 0x7fb7880b5c90 2023-07-06T20:14:10.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78759e700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7880b60e0 0x7fb7880d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb786d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb78815fca0 0x7fb7880d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb786d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb78815fca0 0x7fb7880d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46844/0 (socket says 172.21.15.27:46844) 2023-07-06T20:14:10.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78759e700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7880b60e0 0x7fb7880d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59286/0 (socket says 172.21.15.27:59286) 2023-07-06T20:14:10.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb786d9d700 1 -- 172.21.15.27:0/3550087698 learned_addr learned my addr 172.21.15.27:0/3550087698 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:10.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.206+0000 7fb78659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:51408/0 (socket says 172.21.15.27:51408) 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb78759e700 1 -- 172.21.15.27:0/3550087698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 msgr2=0x7fb7880d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb78759e700 1 --2- 172.21.15.27:0/3550087698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb78759e700 1 -- 172.21.15.27:0/3550087698 >> v2:172.21.15.153:3300/0 conn(0x7fb78815fca0 msgr2=0x7fb7880d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb78759e700 1 --2- 172.21.15.27:0/3550087698 >> v2:172.21.15.153:3300/0 conn(0x7fb78815fca0 0x7fb7880d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb78759e700 1 -- 172.21.15.27:0/3550087698 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880cda50 con 0x7fb7880b60e0 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb78759e700 1 --2- 172.21.15.27:0/3550087698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7880b60e0 0x7fb7880d3280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb7880d0930 tx=0x7fb78000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8acc8ac4caf53f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb785d9b700 1 -- 172.21.15.27:0/3550087698 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000e070 con 0x7fb7880b60e0 2023-07-06T20:14:10.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb785d9b700 1 -- 172.21.15.27:0/3550087698 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78000bb20 con 0x7fb7880b60e0 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.207+0000 7fb785d9b700 1 -- 172.21.15.27:0/3550087698 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780012430 con 0x7fb7880b60e0 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3550087698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7880b60e0 msgr2=0x7fb7880d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3550087698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7880b60e0 0x7fb7880d3280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb7880d0930 tx=0x7fb78000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3550087698 shutdown_connections 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3550087698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7880b60e0 0x7fb7880d3280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3550087698 >> v2:172.21.15.153:3300/0 conn(0x7fb78815fca0 0x7fb7880d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3550087698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3550087698 >> 172.21.15.27:0/3550087698 conn(0x7fb78806c150 msgr2=0x7fb78806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:10.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3550087698 shutdown_connections 2023-07-06T20:14:10.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.208+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3550087698 wait complete. 2023-07-06T20:14:10.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.210+0000 7fb78d1ff700 1 Processor -- start 2023-07-06T20:14:10.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.210+0000 7fb78d1ff700 1 -- start start 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78d1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78d1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7880b60e0 0x7fb7881e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78d1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 0x7fb78815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78d1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7881f2140 con 0x7fb78815fca0 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78d1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb788145f30 con 0x7fb7880b60e0 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78d1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb788146210 con 0x7fb7880b5c90 2023-07-06T20:14:10.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 0x7fb78815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 0x7fb78815ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59296/0 (socket says 172.21.15.27:59296) 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78759e700 1 -- 172.21.15.27:0/3438331465 learned_addr learned my addr 172.21.15.27:0/3438331465 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb78659c700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7880b60e0 0x7fb7881e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.211+0000 7fb786d9d700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78759e700 1 -- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 msgr2=0x7fb7880b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78759e700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78759e700 1 -- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7880b60e0 msgr2=0x7fb7881e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78759e700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7880b60e0 0x7fb7881e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78759e700 1 -- 172.21.15.27:0/3438331465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb780009a70 con 0x7fb78815fca0 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb786d9d700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78759e700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 0x7fb78815ba40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb780014040 tx=0x7fb780004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800048b0 con 0x7fb78815fca0 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780020070 con 0x7fb78815fca0 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7881464f0 con 0x7fb78815fca0 2023-07-06T20:14:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800276c0 con 0x7fb78815fca0 2023-07-06T20:14:10.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.212+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb788146ab0 con 0x7fb78815fca0 2023-07-06T20:14:10.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.213+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7880008d0 con 0x7fb78815fca0 2023-07-06T20:14:10.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.214+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb780012430 con 0x7fb78815fca0 2023-07-06T20:14:10.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.215+0000 7fb76f7fe700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77406ace0 0x7fb77406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.216+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb780022070 con 0x7fb78815fca0 2023-07-06T20:14:10.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.216+0000 7fb786d9d700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77406ace0 0x7fb77406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.217+0000 7fb786d9d700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77406ace0 0x7fb77406d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb778009b80 tx=0x7fb778008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:10.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.220+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb7800564f0 con 0x7fb78815fca0 2023-07-06T20:14:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.491+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb78806a540 con 0x7fb78815fca0 2023-07-06T20:14:10.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.494+0000 7fb76f7fe700 1 -- 172.21.15.27:0/3438331465 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170846 (secure 0 0 0) 0x7fb780059b10 con 0x7fb78815fca0 2023-07-06T20:14:10.498 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77406ace0 msgr2=0x7fb77406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77406ace0 0x7fb77406d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb778009b80 tx=0x7fb778008040 comp rx=0 tx=0).stop 2023-07-06T20:14:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 msgr2=0x7fb78815ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 0x7fb78815ba40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb780014040 tx=0x7fb780004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 shutdown_connections 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77406ace0 0x7fb77406d190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78815fca0 0x7fb78815ba40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7880b5c90 0x7fb7880b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 --2- 172.21.15.27:0/3438331465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7880b60e0 0x7fb7881e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 >> 172.21.15.27:0/3438331465 conn(0x7fb78806c150 msgr2=0x7fb7880d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 shutdown_connections 2023-07-06T20:14:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.498+0000 7fb78d1ff700 1 -- 172.21.15.27:0/3438331465 wait complete. 2023-07-06T20:14:10.518 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":310108246},{"type":"v1","addr":"172.21.15.153:6833","nonce":310108246}]},"active_addr":"172.21.15.153:6833/310108246","active_change":"2023-07-06T20:13:38.327185+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2008626989}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":1233037181}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2859005037}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":3315721707}]}]} 2023-07-06T20:14:10.522 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-06T20:14:10.523 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-06T20:14:10.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:10.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.724+0000 7f1dc7dff700 1 Processor -- start 2023-07-06T20:14:10.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.724+0000 7f1dc7dff700 1 -- start start 2023-07-06T20:14:10.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.725+0000 7f1dc7dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.725+0000 7f1dc7dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1dc00b5c90 0x7f1dc00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.725+0000 7f1dc7dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc00cf780 con 0x7f1dc00b6100 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.725+0000 7f1dc7dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc00cd490 con 0x7f1dc00b5c90 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.725+0000 7f1dc7dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00cd770 con 0x7f1dc015fca0 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1dc00b5c90 0x7f1dc00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1dc00b5c90 0x7f1dc00d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59298/0 (socket says 172.21.15.27:59298) 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 -- 172.21.15.27:0/3879323782 learned_addr learned my addr 172.21.15.27:0/3879323782 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc639c700 1 --1- 172.21.15.27:0/3879323782 >> v1:172.21.15.27:6789/0 conn(0x7f1dc00b6100 0x7f1dc00cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46108/0 (socket says 172.21.15.27:46108) 2023-07-06T20:14:10.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc5b9b700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 -- 172.21.15.27:0/3879323782 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 msgr2=0x7f1dc00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 -- 172.21.15.27:0/3879323782 >> v1:172.21.15.27:6789/0 conn(0x7f1dc00b6100 legacy=0x7f1dc00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 -- 172.21.15.27:0/3879323782 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00cda50 con 0x7f1dc00b5c90 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc5b9b700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc4b99700 1 -- 172.21.15.27:0/3879323782 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630479932 0 0) 0x7f1dc00cf780 con 0x7f1dc00b6100 2023-07-06T20:14:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.726+0000 7f1dc539a700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc00d2d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1db00098f0 tx=0x7f1db0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eead79fa4393f739 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc4b99700 1 -- 172.21.15.27:0/3879323782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000b8a0 con 0x7f1dc00b5c90 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc4b99700 1 -- 172.21.15.27:0/3879323782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db000ba00 con 0x7f1dc00b5c90 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc4b99700 1 -- 172.21.15.27:0/3879323782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000bd90 con 0x7f1dc00b5c90 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 -- 172.21.15.27:0/3879323782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 msgr2=0x7f1dc00d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc00d2d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1db00098f0 tx=0x7f1db0009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 -- 172.21.15.27:0/3879323782 shutdown_connections 2023-07-06T20:14:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/3879323782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc00d2d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 -- 172.21.15.27:0/3879323782 >> 172.21.15.27:0/3879323782 conn(0x7f1dc006c150 msgr2=0x7f1dc006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:10.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 -- 172.21.15.27:0/3879323782 shutdown_connections 2023-07-06T20:14:10.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.727+0000 7f1dc7dff700 1 -- 172.21.15.27:0/3879323782 wait complete. 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.729+0000 7f1dc7dff700 1 Processor -- start 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.729+0000 7f1dc7dff700 1 -- start start 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc7dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc0156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc7dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc00b6100 0x7f1dc01e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc7dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc01565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc7dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc01575c0 con 0x7f1dc00b5c90 2023-07-06T20:14:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc7dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0156b30 con 0x7f1dc00b6100 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc7dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc01e65c0 con 0x7f1dc015fca0 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc5b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc0156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc5b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc0156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59312/0 (socket says 172.21.15.27:59312) 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.730+0000 7f1dc5b9b700 1 -- 172.21.15.27:0/4096813807 learned_addr learned my addr 172.21.15.27:0/4096813807 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc539a700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc00b6100 0x7f1dc01e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc639c700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc01565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc5b9b700 1 -- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 msgr2=0x7f1dc01565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc5b9b700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc01565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc5b9b700 1 -- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc00b6100 msgr2=0x7f1dc01e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc5b9b700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc00b6100 0x7f1dc01e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc5b9b700 1 -- 172.21.15.27:0/4096813807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db0009550 con 0x7f1dc00b5c90 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc639c700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc01565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc5b9b700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc0156cc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00e810 tx=0x7f1dbc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc00c020 con 0x7f1dc00b5c90 2023-07-06T20:14:10.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc004250 con 0x7f1dc00b5c90 2023-07-06T20:14:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0036a0 con 0x7f1dc00b5c90 2023-07-06T20:14:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.731+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc01e68a0 con 0x7f1dc00b5c90 2023-07-06T20:14:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.732+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc01e6e60 con 0x7f1dc00b5c90 2023-07-06T20:14:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.732+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc01e74b0 con 0x7f1dc00b5c90 2023-07-06T20:14:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.734+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1dbc003800 con 0x7f1dc00b5c90 2023-07-06T20:14:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.735+0000 7f1db67fc700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1da406ace0 0x7f1da406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.735+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1dbc088e40 con 0x7f1dc00b5c90 2023-07-06T20:14:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.735+0000 7f1dc539a700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1da406ace0 0x7f1da406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.736+0000 7f1dc539a700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1da406ace0 0x7f1da406d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009c40 tx=0x7f1db0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:10.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.739+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1dbc053f60 con 0x7f1dc00b5c90 2023-07-06T20:14:10.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.949+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dc00008d0 con 0x7f1dc00b5c90 2023-07-06T20:14:10.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.950+0000 7f1db67fc700 1 -- 172.21.15.27:0/4096813807 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f1dbc057580 con 0x7f1dc00b5c90 2023-07-06T20:14:10.952 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:10.952 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:10.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1da406ace0 msgr2=0x7f1da406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1da406ace0 0x7f1da406d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009c40 tx=0x7f1db0008030 comp rx=0 tx=0).stop 2023-07-06T20:14:10.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 msgr2=0x7f1dc0156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc0156cc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00e810 tx=0x7f1dbc00eb20 comp rx=0 tx=0).stop 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 shutdown_connections 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1da406ace0 0x7f1da406d190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1dc015fca0 0x7f1dc01565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dc00b5c90 0x7f1dc0156cc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 --2- 172.21.15.27:0/4096813807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1dc00b6100 0x7f1dc01e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 >> 172.21.15.27:0/4096813807 conn(0x7f1dc006c150 msgr2=0x7f1dc006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:10.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.952+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 shutdown_connections 2023-07-06T20:14:10.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:10.953+0000 7f1dc7dff700 1 -- 172.21.15.27:0/4096813807 wait complete. 2023-07-06T20:14:10.972 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-06T20:14:10.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:11.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.175+0000 7f30f83ff700 1 Processor -- start 2023-07-06T20:14:11.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.175+0000 7f30f83ff700 1 -- start start 2023-07-06T20:14:11.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f83ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30f015fca0 0x7f30f00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f83ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f30f00b6110 0x7f30f00cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f83ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f00cf780 con 0x7f30f00b5c90 2023-07-06T20:14:11.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f83ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f00cd490 con 0x7f30f015fca0 2023-07-06T20:14:11.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f83ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f00cd770 con 0x7f30f00b6110 2023-07-06T20:14:11.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f619b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30f015fca0 0x7f30f00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f599a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f30f00b5c90 0x7f30f00d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46122/0 (socket says 172.21.15.27:46122) 2023-07-06T20:14:11.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.176+0000 7f30f599a700 1 -- 172.21.15.27:0/3400064142 learned_addr learned my addr 172.21.15.27:0/3400064142 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:11.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f699c700 1 --2- 172.21.15.27:0/3400064142 >> v2:172.21.15.153:3300/0 conn(0x7f30f00b6110 0x7f30f00cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f619b700 1 -- 172.21.15.27:0/3400064142 >> v2:172.21.15.153:3300/0 conn(0x7f30f00b6110 msgr2=0x7f30f00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f619b700 1 --2- 172.21.15.27:0/3400064142 >> v2:172.21.15.153:3300/0 conn(0x7f30f00b6110 0x7f30f00cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f619b700 1 -- 172.21.15.27:0/3400064142 >> v1:172.21.15.27:6789/0 conn(0x7f30f00b5c90 legacy=0x7f30f00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f619b700 1 -- 172.21.15.27:0/3400064142 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00cda50 con 0x7f30f015fca0 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f619b700 1 --2- 172.21.15.27:0/3400064142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00d1420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30f00cd410 tx=0x7f30ec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccb9d95047f2b42 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f5199700 1 -- 172.21.15.27:0/3400064142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec011830 con 0x7f30f015fca0 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f5199700 1 -- 172.21.15.27:0/3400064142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec011990 con 0x7f30f015fca0 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.177+0000 7f30f5199700 1 -- 172.21.15.27:0/3400064142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec011d20 con 0x7f30f015fca0 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 -- 172.21.15.27:0/3400064142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 msgr2=0x7f30f00d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 --2- 172.21.15.27:0/3400064142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00d1420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f30f00cd410 tx=0x7f30ec009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 -- 172.21.15.27:0/3400064142 shutdown_connections 2023-07-06T20:14:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 --2- 172.21.15.27:0/3400064142 >> v2:172.21.15.153:3300/0 conn(0x7f30f00b6110 0x7f30f00cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 --2- 172.21.15.27:0/3400064142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00d1420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 -- 172.21.15.27:0/3400064142 >> 172.21.15.27:0/3400064142 conn(0x7f30f006c150 msgr2=0x7f30f006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 -- 172.21.15.27:0/3400064142 shutdown_connections 2023-07-06T20:14:11.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.178+0000 7f30f83ff700 1 -- 172.21.15.27:0/3400064142 wait complete. 2023-07-06T20:14:11.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.180+0000 7f30f83ff700 1 Processor -- start 2023-07-06T20:14:11.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.180+0000 7f30f83ff700 1 -- start start 2023-07-06T20:14:11.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f83ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f00b5c90 0x7f30f01577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f83ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30f00b6110 0x7f30f015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f83ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f83ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0157d20 con 0x7f30f015fca0 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f83ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00cb3a0 con 0x7f30f00b5c90 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f83ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f00cb680 con 0x7f30f00b6110 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f699c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f699c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59332/0 (socket says 172.21.15.27:59332) 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f699c700 1 -- 172.21.15.27:0/489941484 learned_addr learned my addr 172.21.15.27:0/489941484 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f619b700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f00b5c90 0x7f30f01577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f599a700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30f00b6110 0x7f30f015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.181+0000 7f30f699c700 1 -- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30f00b6110 msgr2=0x7f30f015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f699c700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30f00b6110 0x7f30f015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f699c700 1 -- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f00b5c90 msgr2=0x7f30f01577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f699c700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f00b5c90 0x7f30f01577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f699c700 1 -- 172.21.15.27:0/489941484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec0094a0 con 0x7f30f015fca0 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f619b700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f00b5c90 0x7f30f01577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f699c700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00cc440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f30e8007e40 tx=0x7f30e800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800eea0 con 0x7f30f015fca0 2023-07-06T20:14:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e8014420 con 0x7f30f015fca0 2023-07-06T20:14:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f00cb960 con 0x7f30f015fca0 2023-07-06T20:14:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80147b0 con 0x7f30f015fca0 2023-07-06T20:14:11.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.182+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f00cbf20 con 0x7f30f015fca0 2023-07-06T20:14:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.183+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f00008d0 con 0x7f30f015fca0 2023-07-06T20:14:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.184+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30e8014910 con 0x7f30f015fca0 2023-07-06T20:14:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.185+0000 7f30e6ffd700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f30d406ace0 0x7f30d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.185+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30e8089cf0 con 0x7f30f015fca0 2023-07-06T20:14:11.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.186+0000 7f30f619b700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f30d406ace0 0x7f30d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.186+0000 7f30f619b700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f30d406ace0 0x7f30d406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30f00cfff0 tx=0x7f30ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.190+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30e8054d00 con 0x7f30f015fca0 2023-07-06T20:14:11.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.398+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30f00cfd40 con 0x7f30f015fca0 2023-07-06T20:14:11.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.399+0000 7f30e6ffd700 1 -- 172.21.15.27:0/489941484 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f30e8058320 con 0x7f30f015fca0 2023-07-06T20:14:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:11.402 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:11.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.401+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f30d406ace0 msgr2=0x7f30d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.401+0000 7f30f83ff700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f30d406ace0 0x7f30d406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30f00cfff0 tx=0x7f30ec008030 comp rx=0 tx=0).stop 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.401+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 msgr2=0x7f30f00cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.401+0000 7f30f83ff700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00cc440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f30e8007e40 tx=0x7f30e800e3c0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.401+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 shutdown_connections 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.401+0000 7f30f83ff700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f30d406ace0 0x7f30d406d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.402+0000 7f30f83ff700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30f015fca0 0x7f30f00cc440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.402+0000 7f30f83ff700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f00b5c90 0x7f30f01577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.402+0000 7f30f83ff700 1 --2- 172.21.15.27:0/489941484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f30f00b6110 0x7f30f015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.402+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 >> 172.21.15.27:0/489941484 conn(0x7f30f006c150 msgr2=0x7f30f015a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.402+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 shutdown_connections 2023-07-06T20:14:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.402+0000 7f30f83ff700 1 -- 172.21.15.27:0/489941484 wait complete. 2023-07-06T20:14:11.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-06T20:14:11.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-06T20:14:11.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-06T20:14:11.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-06T20:14:11.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-06T20:14:11.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-06T20:14:11.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-06T20:14:11.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-06T20:14:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.790+0000 7f6df57ff700 1 Processor -- start 2023-07-06T20:14:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.790+0000 7f6df57ff700 1 -- start start 2023-07-06T20:14:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.793+0000 7f6a829ff700 1 Processor -- start 2023-07-06T20:14:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.792+0000 7f6df57ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6df00caeb0 0x7f6df0158610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.792+0000 7f6df57ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 0x7f6df00b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.792+0000 7f6df57ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df006feb0 con 0x7f6df00cbf50 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.792+0000 7f6df57ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df006c780 con 0x7f6df00caeb0 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.792+0000 7f6df57ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00b0d80 con 0x7f6df00b0440 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 0x7f6df00b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 0x7f6df00b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:51444/0 (socket says 172.21.15.27:51444) 2023-07-06T20:14:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 -- 172.21.15.27:0/186210802 learned_addr learned my addr 172.21.15.27:0/186210802 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.793+0000 7f6a829ff700 1 -- start start 2023-07-06T20:14:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 -- 172.21.15.27:0/186210802 >> v2:172.21.15.27:3300/0 conn(0x7f6df00caeb0 msgr2=0x7f6df0158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 --2- 172.21.15.27:0/186210802 >> v2:172.21.15.27:3300/0 conn(0x7f6df00caeb0 0x7f6df0158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 -- 172.21.15.27:0/186210802 >> v1:172.21.15.153:6789/0 conn(0x7f6df00cbf50 legacy=0x7f6df006bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.794+0000 7f6def7fe700 1 -- 172.21.15.27:0/186210802 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df006f860 con 0x7f6df00b0440 2023-07-06T20:14:11.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.795+0000 7f6def7fe700 1 --2- 172.21.15.27:0/186210802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 0x7f6df00b0840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098f0 tx=0x7f6de4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0b36f3165c8c53 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.795+0000 7f6a829ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c06b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.795+0000 7f6a829ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b0440 0x7f6a7c0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.795+0000 7f6a829ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0b0dd0 con 0x7f6a7c164530 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.795+0000 7f6a829ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0cc3b0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.795+0000 7f6a829ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1649f0 con 0x7f6a7c160900 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b0440 0x7f6a7c0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b0440 0x7f6a7c0b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46894/0 (socket says 172.21.15.27:46894) 2023-07-06T20:14:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 -- 172.21.15.27:0/3683869158 learned_addr learned my addr 172.21.15.27:0/3683869158 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a7bfff700 1 --2- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c06b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 -- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 msgr2=0x7f6a7c06b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 --2- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c06b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 -- 172.21.15.27:0/3683869158 >> v1:172.21.15.153:6789/0 conn(0x7f6a7c164530 legacy=0x7f6a7c0adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.796+0000 7f6a80f9c700 1 -- 172.21.15.27:0/3683869158 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c164cd0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.797+0000 7f6a80f9c700 1 --2- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0b0890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a740098f0 tx=0x7f6a74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39136b1f6f6ed373 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.797+0000 7f6a7affd700 1 -- 172.21.15.27:0/3683869158 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400b8a0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.797+0000 7f6a7affd700 1 -- 172.21.15.27:0/3683869158 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7400ba00 con 0x7f6a7c0b0440 2023-07-06T20:14:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.797+0000 7f6a7affd700 1 -- 172.21.15.27:0/3683869158 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400bd90 con 0x7f6a7c0b0440 2023-07-06T20:14:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.798+0000 7f6a829ff700 1 -- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b0440 msgr2=0x7f6a7c0b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.798+0000 7f6a829ff700 1 --2- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0b0890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a740098f0 tx=0x7f6a74009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6a829ff700 1 -- 172.21.15.27:0/3683869158 shutdown_connections 2023-07-06T20:14:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6a829ff700 1 --2- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c06b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6a829ff700 1 --2- 172.21.15.27:0/3683869158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0b0890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6a829ff700 1 -- 172.21.15.27:0/3683869158 >> 172.21.15.27:0/3683869158 conn(0x7f6a7c0cb440 msgr2=0x7f6a7c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.798+0000 7f6dedffb700 1 -- 172.21.15.27:0/186210802 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4004230 con 0x7f6df00b0440 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.798+0000 7f6dedffb700 1 -- 172.21.15.27:0/186210802 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4004390 con 0x7f6df00b0440 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.798+0000 7f6dedffb700 1 -- 172.21.15.27:0/186210802 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40036a0 con 0x7f6df00b0440 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6df57ff700 1 -- 172.21.15.27:0/186210802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 msgr2=0x7f6df00b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6df57ff700 1 --2- 172.21.15.27:0/186210802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 0x7f6df00b0840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098f0 tx=0x7f6de4009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6df57ff700 1 -- 172.21.15.27:0/186210802 shutdown_connections 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6df57ff700 1 --2- 172.21.15.27:0/186210802 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b0440 0x7f6df00b0840 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6df57ff700 1 --2- 172.21.15.27:0/186210802 >> v2:172.21.15.27:3300/0 conn(0x7f6df00caeb0 0x7f6df0158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.799+0000 7f6df57ff700 1 -- 172.21.15.27:0/186210802 >> 172.21.15.27:0/186210802 conn(0x7f6df00cb440 msgr2=0x7f6df00cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.800+0000 7f6a829ff700 1 -- 172.21.15.27:0/3683869158 shutdown_connections 2023-07-06T20:14:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.800+0000 7f6a829ff700 1 -- 172.21.15.27:0/3683869158 wait complete. 2023-07-06T20:14:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.800+0000 7f6df57ff700 1 -- 172.21.15.27:0/186210802 shutdown_connections 2023-07-06T20:14:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.801+0000 7f6df57ff700 1 -- 172.21.15.27:0/186210802 wait complete. 2023-07-06T20:14:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.803+0000 7f6a829ff700 1 Processor -- start 2023-07-06T20:14:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.803+0000 7f6a829ff700 1 -- start start 2023-07-06T20:14:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.803+0000 7f6df57ff700 1 Processor -- start 2023-07-06T20:14:11.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.803+0000 7f6df57ff700 1 -- start start 2023-07-06T20:14:11.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.805+0000 7f6df57ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b0440 0x7f6df01695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.805+0000 7f6df57ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00caeb0 0x7f6df015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.805+0000 7f6df57ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00cbf50 0x7f6df00b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.805+0000 7f6df57ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df014b430 con 0x7f6df00caeb0 2023-07-06T20:14:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.805+0000 7f6df57ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df014b150 con 0x7f6df00b0440 2023-07-06T20:14:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.805+0000 7f6df57ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00b2930 con 0x7f6df00cbf50 2023-07-06T20:14:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.806+0000 7f6dee7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00caeb0 0x7f6df015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.806+0000 7f6dee7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00caeb0 0x7f6df015b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59358/0 (socket says 172.21.15.27:59358) 2023-07-06T20:14:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.806+0000 7f6dee7fc700 1 -- 172.21.15.27:0/3237919170 learned_addr learned my addr 172.21.15.27:0/3237919170 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.806+0000 7f6deeffd700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b0440 0x7f6df01695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6def7fe700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00cbf50 0x7f6df00b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6deeffd700 1 -- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00cbf50 msgr2=0x7f6df00b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6deeffd700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00cbf50 0x7f6df00b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6deeffd700 1 -- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00caeb0 msgr2=0x7f6df015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6deeffd700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00caeb0 0x7f6df015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6deeffd700 1 -- 172.21.15.27:0/3237919170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4009550 con 0x7f6df00b0440 2023-07-06T20:14:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.808+0000 7f6deeffd700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b0440 0x7f6df01695d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6de800bd50 tx=0x7f6de800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7fa2179ff700 1 Processor -- start 2023-07-06T20:14:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6a829ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6a829ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6a829ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c164530 0x7f6a7c0b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6a829ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b22e0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6a829ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b25b0 con 0x7f6a7c164530 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.807+0000 7f6a829ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b2890 con 0x7f6a7c160900 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.808+0000 7f6a7bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.808+0000 7f6a7bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59360/0 (socket says 172.21.15.27:59360) 2023-07-06T20:14:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.808+0000 7f6a7bfff700 1 -- 172.21.15.27:0/3170676405 learned_addr learned my addr 172.21.15.27:0/3170676405 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.808+0000 7f6a7b7fe700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a80f9c700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c164530 0x7f6a7c0b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a7bfff700 1 -- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 msgr2=0x7f6a7c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a7bfff700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c160900 0x7f6a7c0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a7bfff700 1 -- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c164530 msgr2=0x7f6a7c0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a7bfff700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c164530 0x7f6a7c0b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a7bfff700 1 -- 172.21.15.27:0/3170676405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a74009550 con 0x7f6a7c0b0440 2023-07-06T20:14:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7fa2179ff700 1 -- start start 2023-07-06T20:14:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.809+0000 7f6a7bfff700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0ce550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000e810 tx=0x7f6a7000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70013610 con 0x7f6a7c0b0440 2023-07-06T20:14:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70013770 con 0x7f6a7c0b0440 2023-07-06T20:14:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70013ae0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.810+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800cbb0 con 0x7f6df00b0440 2023-07-06T20:14:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.810+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800cd10 con 0x7f6df00b0440 2023-07-06T20:14:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.810+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8014680 con 0x7f6df00b0440 2023-07-06T20:14:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.810+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df00b2c10 con 0x7f6df00b0440 2023-07-06T20:14:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.810+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df00b3cb0 con 0x7f6df00b0440 2023-07-06T20:14:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7fa2179ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2100caeb0 0x7fa210158610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7fa2179ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2100b0440 0x7fa2100b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7fa2179ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21006feb0 con 0x7fa2100cbf50 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7fa2179ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21006c780 con 0x7fa2100b0440 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.811+0000 7fa2179ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2100b0d50 con 0x7fa2100caeb0 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.812+0000 7fa214f9a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2100caeb0 0x7fa210158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.812+0000 7fa214f9a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa2100caeb0 0x7fa210158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46920/0 (socket says 172.21.15.27:46920) 2023-07-06T20:14:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.812+0000 7fa214f9a700 1 -- 172.21.15.27:0/1599377378 learned_addr learned my addr 172.21.15.27:0/1599377378 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.812+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6dd0000fc0 con 0x7f6df00b0440 2023-07-06T20:14:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.812+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c0b2b70 con 0x7f6a7c0b0440 2023-07-06T20:14:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.813+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0b3130 con 0x7f6a7c0b0440 2023-07-06T20:14:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.813+0000 7fa214f9a700 1 -- 172.21.15.27:0/1599377378 >> v2:172.21.15.27:3300/0 conn(0x7fa2100b0440 msgr2=0x7fa2100b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.813+0000 7fa214f9a700 1 --2- 172.21.15.27:0/1599377378 >> v2:172.21.15.27:3300/0 conn(0x7fa2100b0440 0x7fa2100b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.813+0000 7fa214f9a700 1 -- 172.21.15.27:0/1599377378 >> v1:172.21.15.153:6789/0 conn(0x7fa2100cbf50 legacy=0x7fa21006bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.813+0000 7fa214f9a700 1 -- 172.21.15.27:0/1599377378 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21006f860 con 0x7fa2100caeb0 2023-07-06T20:14:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.813+0000 7fa214f9a700 1 --2- 172.21.15.27:0/1599377378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 0x7fa210158610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa21015d400 tx=0x7fa20c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b045119a4f5429 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.814+0000 7fa207fff700 1 -- 172.21.15.27:0/1599377378 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004230 con 0x7fa2100caeb0 2023-07-06T20:14:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.814+0000 7fa207fff700 1 -- 172.21.15.27:0/1599377378 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c004390 con 0x7fa2100caeb0 2023-07-06T20:14:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.814+0000 7fa207fff700 1 -- 172.21.15.27:0/1599377378 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0036a0 con 0x7fa2100caeb0 2023-07-06T20:14:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a7c165290 con 0x7f6a7c0b0440 2023-07-06T20:14:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 -- 172.21.15.27:0/1599377378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 msgr2=0x7fa210158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 --2- 172.21.15.27:0/1599377378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 0x7fa210158610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa21015d400 tx=0x7fa20c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 -- 172.21.15.27:0/1599377378 shutdown_connections 2023-07-06T20:14:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 --2- 172.21.15.27:0/1599377378 >> v2:172.21.15.27:3300/0 conn(0x7fa2100b0440 0x7fa2100b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 --2- 172.21.15.27:0/1599377378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 0x7fa210158610 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 -- 172.21.15.27:0/1599377378 >> 172.21.15.27:0/1599377378 conn(0x7fa2100cb440 msgr2=0x7fa2100cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 -- 172.21.15.27:0/1599377378 shutdown_connections 2023-07-06T20:14:11.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7fa2179ff700 1 -- 172.21.15.27:0/1599377378 wait complete. 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.814+0000 7f62037ff700 1 Processor -- start 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.815+0000 7f62037ff700 1 -- start start 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.817+0000 7f62037ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f61fc0b0b50 0x7f61fc0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.817+0000 7f62037ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0ca930 0x7f61fc0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.817+0000 7f62037ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc06bb30 con 0x7f61fc070db0 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.817+0000 7f62037ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc06b220 con 0x7f61fc0b0b50 2023-07-06T20:14:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.817+0000 7f62037ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0cb140 con 0x7f61fc0ca930 2023-07-06T20:14:11.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.816+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6de801d3b0 con 0x7f6df00b0440 2023-07-06T20:14:11.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6dd77fe700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6ddc06adb0 0x7f6ddc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6de80897c0 con 0x7f6df00b0440 2023-07-06T20:14:11.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6201d9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0ca930 0x7f61fc0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6201d9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0ca930 0x7f61fc0ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:51492/0 (socket says 172.21.15.27:51492) 2023-07-06T20:14:11.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6201d9c700 1 -- 172.21.15.27:0/2664633001 learned_addr learned my addr 172.21.15.27:0/2664633001 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6dd77fe700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7f6ddc073660 0x7f6ddc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 --> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ddc076120 con 0x7f6ddc073660 2023-07-06T20:14:11.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6de8089b70 con 0x7f6df00b0440 2023-07-06T20:14:11.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6200d9a700 1 --2- 172.21.15.27:0/2664633001 >> v2:172.21.15.27:3300/0 conn(0x7f61fc0b0b50 0x7f61fc0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.819+0000 7f6dee7fc700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6ddc06adb0 0x7f6ddc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.819+0000 7f6def7fe700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7f6ddc073660 0x7f6ddc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7fa2179ff700 1 Processor -- start 2023-07-06T20:14:11.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.819+0000 7f6200d9a700 1 -- 172.21.15.27:0/2664633001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0ca930 msgr2=0x7f61fc0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.819+0000 7f6200d9a700 1 --2- 172.21.15.27:0/2664633001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0ca930 0x7f61fc0ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.819+0000 7f6200d9a700 1 -- 172.21.15.27:0/2664633001 >> v1:172.21.15.27:6789/0 conn(0x7f61fc070db0 legacy=0x7f61fc0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.819+0000 7f6200d9a700 1 -- 172.21.15.27:0/2664633001 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc06b4e0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.818+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a70013c40 con 0x7f6a7c0b0440 2023-07-06T20:14:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6a78ff9700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6806adb0 0x7f6a6806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a700890b0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6a78ff9700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f6a68073660 0x7f6a68075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 --> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a68076120 con 0x7f6a68073660 2023-07-06T20:14:11.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a7008b6d0 con 0x7f6a7c0b0440 2023-07-06T20:14:11.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6def7fe700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7f6ddc073660 0x7f6ddc075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.820+0000 7f6200d9a700 1 --2- 172.21.15.27:0/2664633001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0b0b50 0x7f61fc0ca3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009d90 tx=0x7f61f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853b16d13fc3df9e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f61f3fff700 1 -- 172.21.15.27:0/2664633001 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400e070 con 0x7f61fc0b0b50 2023-07-06T20:14:11.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f61f3fff700 1 -- 172.21.15.27:0/2664633001 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f400bad0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f61f3fff700 1 -- 172.21.15.27:0/2664633001 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400be80 con 0x7f61fc0b0b50 2023-07-06T20:14:11.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f6a7b7fe700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6806adb0 0x7f6a6806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f6a80f9c700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f6a68073660 0x7f6a68075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== osd.1 v2:172.21.15.27:6824/3450674250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6ddc076120 con 0x7f6ddc073660 2023-07-06T20:14:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.821+0000 7f96657ff700 1 Processor -- start 2023-07-06T20:14:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.822+0000 7f62037ff700 1 -- 172.21.15.27:0/2664633001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0b0b50 msgr2=0x7f61fc0ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.822+0000 7f62037ff700 1 --2- 172.21.15.27:0/2664633001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0b0b50 0x7f61fc0ca3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009d90 tx=0x7f61f400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.822+0000 7f96657ff700 1 -- start start 2023-07-06T20:14:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f6a7b7fe700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6806adb0 0x7f6a6806d260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cf7a0 tx=0x7f6a6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f6a80f9c700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f6a68073660 0x7f6a68075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f62037ff700 1 -- 172.21.15.27:0/2664633001 shutdown_connections 2023-07-06T20:14:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f62037ff700 1 --2- 172.21.15.27:0/2664633001 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0ca930 0x7f61fc0ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f62037ff700 1 --2- 172.21.15.27:0/2664633001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0b0b50 0x7f61fc0ca3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f62037ff700 1 -- 172.21.15.27:0/2664633001 >> 172.21.15.27:0/2664633001 conn(0x7f61fc0cb440 msgr2=0x7f61fc0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== osd.0 v2:172.21.15.27:6800/323630915 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6a68076120 con 0x7f6a68073660 2023-07-06T20:14:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.823+0000 7f62037ff700 1 -- 172.21.15.27:0/2664633001 shutdown_connections 2023-07-06T20:14:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7f96657ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 0x7f966006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7f96657ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96600cc3b0 con 0x7f96600b0440 2023-07-06T20:14:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7f96657ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96601649f0 con 0x7f9660160900 2023-07-06T20:14:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7f96657ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9660164cd0 con 0x7f9660164530 2023-07-06T20:14:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7f965effd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9660160900 0x7f96600adb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:39358/0 (socket says 172.21.15.27:39358) 2023-07-06T20:14:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7f965effd700 1 -- 172.21.15.27:0/1885001099 learned_addr learned my addr 172.21.15.27:0/1885001099 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7f965e7fc700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 0x7f966006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101481558 0 0) 0x7f96601649f0 con 0x7f9660160900 2023-07-06T20:14:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96480035b0 con 0x7f9660160900 2023-07-06T20:14:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7fa2179ff700 1 -- start start 2023-07-06T20:14:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7fa2179ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2100b0440 0x7fa2100bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7fa2179ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 0x7fa21014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.825+0000 7fa2179ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2100cbf50 0x7fa2100c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7fa2179ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100b3470 con 0x7fa2100cbf50 2023-07-06T20:14:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7fa2179ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100b3750 con 0x7fa2100caeb0 2023-07-06T20:14:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7fa2179ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100c5b30 con 0x7fa2100b0440 2023-07-06T20:14:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 56752393 0 0) 0x7f96480035b0 con 0x7f9660160900 2023-07-06T20:14:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 msgr2=0x7f966006b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 0x7f966006b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 >> v1:172.21.15.27:6789/0 conn(0x7f96600b0440 legacy=0x7f96600ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660164fb0 con 0x7f9660160900 2023-07-06T20:14:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f9654001670 con 0x7f9660160900 2023-07-06T20:14:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 >> v1:172.21.15.153:6789/0 conn(0x7f9660160900 legacy=0x7f96600adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648003a00 0x7f9648003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9648004430 0x7f96480048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 0x7f9648005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660164cd0 con 0x7f9648003a00 2023-07-06T20:14:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bac0 con 0x7f9648004430 2023-07-06T20:14:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bc80 con 0x7f9660164530 2023-07-06T20:14:11.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.824+0000 7f62037ff700 1 -- 172.21.15.27:0/2664633001 wait complete. 2023-07-06T20:14:11.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.826+0000 7f62037ff700 1 Processor -- start 2023-07-06T20:14:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.827+0000 7fa215f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2100cbf50 0x7fa2100c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.827+0000 7fa215f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2100cbf50 0x7fa2100c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59388/0 (socket says 172.21.15.27:59388) 2023-07-06T20:14:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.827+0000 7fa215f9c700 1 -- 172.21.15.27:0/1554373322 learned_addr learned my addr 172.21.15.27:0/1554373322 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa214f9a700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 0x7fa21014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa21579b700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2100b0440 0x7fa2100bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa215f9c700 1 -- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2100b0440 msgr2=0x7fa2100bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa215f9c700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2100b0440 0x7fa2100bf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa215f9c700 1 -- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 msgr2=0x7fa21014ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa215f9c700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2100caeb0 0x7fa21014ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7fa215f9c700 1 -- 172.21.15.27:0/1554373322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c009550 con 0x7fa2100cbf50 2023-07-06T20:14:11.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.827+0000 7f62037ff700 1 -- start start 2023-07-06T20:14:11.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f62037ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61fc070db0 0x7f61fc1e1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f62037ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0b0b50 0x7f61fc0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f62037ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0ca930 0x7f61fc0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f62037ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc2022b0 con 0x7f61fc0ca930 2023-07-06T20:14:11.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f62037ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc158260 con 0x7f61fc070db0 2023-07-06T20:14:11.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f62037ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc1e14b0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7fa215f9c700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2100cbf50 0x7fa2100c6700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b970 tx=0x7fa20800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f6200d9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0b0b50 0x7f61fc0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f6200d9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0b0b50 0x7f61fc0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:51528/0 (socket says 172.21.15.27:51528) 2023-07-06T20:14:11.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f6200d9a700 1 -- 172.21.15.27:0/3142766138 learned_addr learned my addr 172.21.15.27:0/3142766138 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f620159b700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61fc070db0 0x7f61fc1e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20800c9b0 con 0x7fa2100cbf50 2023-07-06T20:14:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20800cb30 con 0x7fa2100cbf50 2023-07-06T20:14:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208014440 con 0x7fa2100cbf50 2023-07-06T20:14:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2100c5e10 con 0x7fa2100cbf50 2023-07-06T20:14:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2100bfcf0 con 0x7fa2100cbf50 2023-07-06T20:14:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f6201d9c700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0ca930 0x7f61fc0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f6200d9a700 1 -- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61fc070db0 msgr2=0x7f61fc1e1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f6200d9a700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61fc070db0 0x7f61fc1e1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f6200d9a700 1 -- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0ca930 msgr2=0x7f61fc0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f6200d9a700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61fc0ca930 0x7f61fc0b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f6200d9a700 1 -- 172.21.15.27:0/3142766138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4009a70 con 0x7f61fc0b0b50 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.827+0000 7f965effd700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648003a00 0x7f9648003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.827+0000 7f965f7fe700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 0x7f9648005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f965effd700 1 -- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 msgr2=0x7f9648005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f965effd700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9660164530 0x7f9648005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f965effd700 1 -- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9648004430 msgr2=0x7f96480048f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f965effd700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9648004430 0x7f96480048f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f965effd700 1 -- 172.21.15.27:0/1885001099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660164fb0 con 0x7f9648003a00 2023-07-06T20:14:11.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.828+0000 7f965effd700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648003a00 0x7f9648003e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9654007fd0 tx=0x7f9648005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e3ced152af531d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654007bf0 con 0x7f9648003a00 2023-07-06T20:14:11.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965400cdd0 con 0x7f9648003a00 2023-07-06T20:14:11.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.829+0000 7f965dffb700 1 -- 172.21.15.27:0/1885001099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654005b80 con 0x7f9648003a00 2023-07-06T20:14:11.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.832+0000 7f96657ff700 1 -- 172.21.15.27:0/1885001099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648003a00 msgr2=0x7f9648003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.832+0000 7f96657ff700 1 --2- 172.21.15.27:0/1885001099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9648003a00 0x7f9648003e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9654007fd0 tx=0x7f9648005880 comp rx=0 tx=0).stop 2023-07-06T20:14:11.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.831+0000 7fdf0e7ff700 1 Processor -- start 2023-07-06T20:14:11.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.831+0000 7fdf0e7ff700 1 -- start start 2023-07-06T20:14:11.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f492e1ff700 1 Processor -- start 2023-07-06T20:14:11.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.830+0000 7f492e1ff700 1 -- start start 2023-07-06T20:14:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.832+0000 7f492e1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49280cdf00 0x7f492816cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.832+0000 7f492e1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492816de80 con 0x7f492808a760 2023-07-06T20:14:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.832+0000 7f492e1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49280ca560 con 0x7f49280ce2d0 2023-07-06T20:14:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.832+0000 7f492e1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4928169960 con 0x7f49280cdf00 2023-07-06T20:14:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.833+0000 7f965f7fe700 1 -- 172.21.15.27:0/1885001099 reap_dead start 2023-07-06T20:14:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7f6dee7fc700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6ddc06adb0 0x7f6ddc06d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6df014ae30 tx=0x7f6de0009290 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1f0000fc0 con 0x7fa2100cbf50 2023-07-06T20:14:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7f6200d9a700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0b0b50 0x7f61fc0b30a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f61fc06f2d0 tx=0x7f61f4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.833+0000 7f4926ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f49280ce2d0 0x7f49280ca9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:39372/0 (socket says 172.21.15.27:39372) 2023-07-06T20:14:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.833+0000 7f4926ffd700 1 -- 172.21.15.27:0/2618619311 learned_addr learned my addr 172.21.15.27:0/2618619311 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7f49277fe700 1 --2- 172.21.15.27:0/2618619311 >> v2:172.21.15.27:3300/0 conn(0x7f49280cdf00 0x7f492816cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7f49267fc700 1 -- 172.21.15.27:0/2618619311 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616430197 0 0) 0x7f49280ca560 con 0x7f49280ce2d0 2023-07-06T20:14:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7f49267fc700 1 -- 172.21.15.27:0/2618619311 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49100035b0 con 0x7f49280ce2d0 2023-07-06T20:14:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf0e7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 0x7fdf08070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf0e7ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdf080b0440 0x7fdf080b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf0e7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0806bb30 con 0x7fdf080ca3f0 2023-07-06T20:14:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf0e7ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080b0df0 con 0x7fdf080b0440 2023-07-06T20:14:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf0e7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08160580 con 0x7fdf0806d800 2023-07-06T20:14:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf077fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdf080b0440 0x7fdf080b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf077fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdf080b0440 0x7fdf080b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46938/0 (socket says 172.21.15.27:46938) 2023-07-06T20:14:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fdf077fe700 1 -- 172.21.15.27:0/1305285744 learned_addr learned my addr 172.21.15.27:0/1305285744 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7fdf07fff700 1 --2- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 0x7fdf08070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa20801d3b0 con 0x7fa2100cbf50 2023-07-06T20:14:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40045f0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4004750 con 0x7f61fc0b0b50 2023-07-06T20:14:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4012450 con 0x7f61fc0b0b50 2023-07-06T20:14:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fbcfbbff700 1 Processor -- start 2023-07-06T20:14:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.834+0000 7fbcfbbff700 1 -- start start 2023-07-06T20:14:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7f49277fe700 1 -- 172.21.15.27:0/2618619311 >> v1:172.21.15.153:6789/0 conn(0x7f49280ce2d0 legacy=0x7f49280ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7f49277fe700 1 -- 172.21.15.27:0/2618619311 >> v1:172.21.15.27:6789/0 conn(0x7f492808a760 legacy=0x7f492806b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7f49277fe700 1 -- 172.21.15.27:0/2618619311 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492816d8c0 con 0x7f49280cdf00 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7fdf077fe700 1 -- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 msgr2=0x7fdf08070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7fdf077fe700 1 --2- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 0x7fdf08070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7fdf077fe700 1 -- 172.21.15.27:0/1305285744 >> v1:172.21.15.153:6789/0 conn(0x7fdf080ca3f0 legacy=0x7fdf080ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.835+0000 7fdf077fe700 1 -- 172.21.15.27:0/1305285744 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080cb140 con 0x7fdf080b0440 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7fdf077fe700 1 --2- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080b08b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009e50 tx=0x7fdef800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=920386ab02f0cfbf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7fdf06ffd700 1 -- 172.21.15.27:0/1305285744 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800e070 con 0x7fdf080b0440 2023-07-06T20:14:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7fdf06ffd700 1 -- 172.21.15.27:0/1305285744 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8003a20 con 0x7fdf080b0440 2023-07-06T20:14:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7fdf06ffd700 1 -- 172.21.15.27:0/1305285744 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8003db0 con 0x7fdf080b0440 2023-07-06T20:14:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7fa205ffb700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa1f406adb0 0x7fa1f406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7f49277fe700 1 --2- 172.21.15.27:0/2618619311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f492816cfc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f492000bd20 tx=0x7f4920010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9823660617a7895 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fbcfbbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d20f0 0x7fbcf41619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fbcfbbff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40d2550 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fbcfbbff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40d27d0 con 0x7fbcf415b1c0 2023-07-06T20:14:11.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fbcfbbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf4161f30 con 0x7fbcf40d20f0 2023-07-06T20:14:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc1e11d0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc0b28a0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fbcf919a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbcf415b1c0 0x7fbcf40d31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:39398/0 (socket says 172.21.15.27:39398) 2023-07-06T20:14:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fbcf919a700 1 -- 172.21.15.27:0/843002940 learned_addr learned my addr 172.21.15.27:0/843002940 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fbcfa19c700 1 --2- 172.21.15.27:0/843002940 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d20f0 0x7fbcf41619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 msgr2=0x7fdf080b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fdf0e7ff700 1 --2- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080b08b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009e50 tx=0x7fdef800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:11.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1305285744 shutdown_connections 2023-07-06T20:14:11.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fdf0e7ff700 1 --2- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 0x7fdf08070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fdf0e7ff700 1 --2- 172.21.15.27:0/1305285744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080b08b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1305285744 >> 172.21.15.27:0/1305285744 conn(0x7fdf080cb440 msgr2=0x7fdf080cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1305285744 shutdown_connections 2023-07-06T20:14:11.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7f96657ff700 1 -- 172.21.15.27:0/1885001099 shutdown_connections 2023-07-06T20:14:11.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7f96657ff700 1 -- 172.21.15.27:0/1885001099 >> 172.21.15.27:0/1885001099 conn(0x7f96600cb440 msgr2=0x7f96600cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.836+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2080891a0 con 0x7fa2100cbf50 2023-07-06T20:14:11.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fa205ffb700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7fa1f40735e0 0x7fa1f40759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 --> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1f40760a0 con 0x7fa1f40735e0 2023-07-06T20:14:11.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.837+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa208057d50 con 0x7fa2100cbf50 2023-07-06T20:14:11.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fbcfa19c700 1 -- 172.21.15.27:0/843002940 >> v1:172.21.15.153:6789/0 conn(0x7fbcf415b1c0 legacy=0x7fbcf40d31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fbcfa19c700 1 -- 172.21.15.27:0/843002940 >> v1:172.21.15.27:6789/0 conn(0x7fbcf40d2dd0 legacy=0x7fbcf40d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fbcfa19c700 1 -- 172.21.15.27:0/843002940 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40d2ab0 con 0x7fbcf40d20f0 2023-07-06T20:14:11.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7fa214f9a700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7fa1f40735e0 0x7fa1f40759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7fa21579b700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa1f406adb0 0x7fa1f406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7fbcfa19c700 1 --2- 172.21.15.27:0/843002940 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d20f0 0x7fbcf41619f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbce40097f0 tx=0x7fbce4009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4aba64b6fba7105 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7f49267fc700 1 -- 172.21.15.27:0/2618619311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920010d10 con 0x7f49280cdf00 2023-07-06T20:14:11.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7f49267fc700 1 -- 172.21.15.27:0/2618619311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4920010e70 con 0x7f49280cdf00 2023-07-06T20:14:11.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7f49267fc700 1 -- 172.21.15.27:0/2618619311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920018620 con 0x7f49280cdf00 2023-07-06T20:14:11.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f492e1ff700 1 -- 172.21.15.27:0/2618619311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 msgr2=0x7f492816cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f492e1ff700 1 --2- 172.21.15.27:0/2618619311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f492816cfc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f492000bd20 tx=0x7f4920010730 comp rx=0 tx=0).stop 2023-07-06T20:14:11.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f96657ff700 1 -- 172.21.15.27:0/1885001099 shutdown_connections 2023-07-06T20:14:11.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f492e1ff700 1 -- 172.21.15.27:0/2618619311 shutdown_connections 2023-07-06T20:14:11.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f492e1ff700 1 --2- 172.21.15.27:0/2618619311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f492816cfc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f492e1ff700 1 -- 172.21.15.27:0/2618619311 >> 172.21.15.27:0/2618619311 conn(0x7f49280cb440 msgr2=0x7f49280cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7fbcf8999700 1 -- 172.21.15.27:0/843002940 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4004230 con 0x7fbcf40d20f0 2023-07-06T20:14:11.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7f96657ff700 1 -- 172.21.15.27:0/1885001099 wait complete. 2023-07-06T20:14:11.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7fbcf8999700 1 -- 172.21.15.27:0/843002940 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce4004390 con 0x7fbcf40d20f0 2023-07-06T20:14:11.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.839+0000 7fbcf8999700 1 -- 172.21.15.27:0/843002940 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce40036a0 con 0x7fbcf40d20f0 2023-07-06T20:14:11.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.840+0000 7fa214f9a700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7fa1f40735e0 0x7fa1f40759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fbcfbbff700 1 -- 172.21.15.27:0/843002940 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d20f0 msgr2=0x7fbcf41619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fbcfbbff700 1 --2- 172.21.15.27:0/843002940 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d20f0 0x7fbcf41619f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbce40097f0 tx=0x7fbce4009b00 comp rx=0 tx=0).stop 2023-07-06T20:14:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.842+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== osd.2 v2:172.21.15.27:6808/380505300 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa1f40760a0 con 0x7fa1f40735e0 2023-07-06T20:14:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61fc06b7c0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fbcfbbff700 1 -- 172.21.15.27:0/843002940 shutdown_connections 2023-07-06T20:14:11.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fbcfbbff700 1 --2- 172.21.15.27:0/843002940 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d20f0 0x7fbcf41619f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fbcfbbff700 1 -- 172.21.15.27:0/843002940 >> 172.21.15.27:0/843002940 conn(0x7fbcf406c150 msgr2=0x7fbcf406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fbcfbbff700 1 -- 172.21.15.27:0/843002940 shutdown_connections 2023-07-06T20:14:11.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.842+0000 7f96657ff700 1 Processor -- start 2023-07-06T20:14:11.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7f492e1ff700 1 -- 172.21.15.27:0/2618619311 shutdown_connections 2023-07-06T20:14:11.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.842+0000 7f492e1ff700 1 -- 172.21.15.27:0/2618619311 wait complete. 2023-07-06T20:14:11.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.838+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1305285744 wait complete. 2023-07-06T20:14:11.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fdf0e7ff700 1 Processor -- start 2023-07-06T20:14:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.841+0000 7fdf0e7ff700 1 -- start start 2023-07-06T20:14:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf0e7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 0x7fdf080cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf0e7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf0e7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080ca3f0 0x7fdf080af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf0e7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080af610 con 0x7fdf080ca3f0 2023-07-06T20:14:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf0e7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08157940 con 0x7fdf080b0440 2023-07-06T20:14:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf0e7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08157dd0 con 0x7fdf0806d800 2023-07-06T20:14:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46952/0 (socket says 172.21.15.27:46952) 2023-07-06T20:14:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fdf077fe700 1 -- 172.21.15.27:0/1231000235 learned_addr learned my addr 172.21.15.27:0/1231000235 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.844+0000 7fdf077fe700 1 -- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 msgr2=0x7fdf080cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61f4012650 con 0x7f61fc0b0b50 2023-07-06T20:14:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7f61f1ffb700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61e006adb0 0x7f61e006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61f408b3a0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7f61f1ffb700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7f61e00735e0 0x7f61e00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 --> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61e00760a0 con 0x7f61e00735e0 2023-07-06T20:14:11.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.843+0000 7fbcfbbff700 1 -- 172.21.15.27:0/843002940 wait complete. 2023-07-06T20:14:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.844+0000 7fdf077fe700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf0806d800 0x7fdf080cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.844+0000 7fdf077fe700 1 -- 172.21.15.27:0/1231000235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080ca3f0 msgr2=0x7fdf080af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.844+0000 7fdf077fe700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080ca3f0 0x7fdf080af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.844+0000 7fdf077fe700 1 -- 172.21.15.27:0/1231000235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8009b30 con 0x7fdf080b0440 2023-07-06T20:14:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7fdf077fe700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080aeb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf0806f2d0 tx=0x7fdef8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7f96657ff700 1 -- start start 2023-07-06T20:14:11.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f6201d9c700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7f61e00735e0 0x7f61e00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004440 con 0x7fdf080b0440 2023-07-06T20:14:11.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef80045a0 con 0x7fdf080b0440 2023-07-06T20:14:11.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004950 con 0x7fdf080b0440 2023-07-06T20:14:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf08156d30 con 0x7fdf080b0440 2023-07-06T20:14:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf081572f0 con 0x7fdf080b0440 2023-07-06T20:14:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f620159b700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61e006adb0 0x7f61e006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f96657ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9648003a00 0x7f96600c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f96657ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c2280 0x7f96600c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f96657ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96600c44b0 0x7f966014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f96657ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96600c2710 con 0x7f96600c2280 2023-07-06T20:14:11.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f96657ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96600c29f0 con 0x7f96600c44b0 2023-07-06T20:14:11.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f96657ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96600c2cd0 con 0x7f9648003a00 2023-07-06T20:14:11.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f965effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9648003a00 0x7f96600c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f965effd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9648003a00 0x7f96600c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:51560/0 (socket says 172.21.15.27:51560) 2023-07-06T20:14:11.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f965effd700 1 -- 172.21.15.27:0/2236871233 learned_addr learned my addr 172.21.15.27:0/2236871233 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.844+0000 7f492e1ff700 1 Processor -- start 2023-07-06T20:14:11.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.845+0000 7f492e1ff700 1 -- start start 2023-07-06T20:14:11.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f492e1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f492808a760 0x7f4928162c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f492e1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f4928160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f492e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49280ce2d0 0x7f4928161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f492e1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928163ef0 con 0x7f49280cdf00 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f492e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280c1ca0 con 0x7f49280ce2d0 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7f492e1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49280c8c00 con 0x7f492808a760 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f49277fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f492808a760 0x7f4928162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f4926ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f4928160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f4926ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f4928160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59422/0 (socket says 172.21.15.27:59422) 2023-07-06T20:14:11.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f4926ffd700 1 -- 172.21.15.27:0/4042922241 learned_addr learned my addr 172.21.15.27:0/4042922241 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.847+0000 7f4927fff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49280ce2d0 0x7f4928161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f965e7fc700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c2280 0x7f96600c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f965f7fe700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96600c44b0 0x7f966014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f4927fff700 1 -- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f492808a760 msgr2=0x7f4928162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f4927fff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f492808a760 0x7f4928162c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f4927fff700 1 -- 172.21.15.27:0/4042922241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 msgr2=0x7f4928160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f4927fff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49280cdf00 0x7f4928160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f4927fff700 1 -- 172.21.15.27:0/4042922241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492000b980 con 0x7f49280ce2d0 2023-07-06T20:14:11.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7f49277fe700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f492808a760 0x7f4928162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:11.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f4927fff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49280ce2d0 0x7f4928161110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c390 tx=0x7f491c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.848+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf0806b7c0 con 0x7fdf080b0440 2023-07-06T20:14:11.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f965e7fc700 1 -- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9648003a00 msgr2=0x7f96600c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f965e7fc700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9648003a00 0x7f96600c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f965e7fc700 1 -- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96600c44b0 msgr2=0x7f966014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f965e7fc700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96600c44b0 0x7f966014c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f965e7fc700 1 -- 172.21.15.27:0/2236871233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96600c2fb0 con 0x7f96600c2280 2023-07-06T20:14:11.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.846+0000 7fa21579b700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa1f406adb0 0x7fa1f406d260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa210169610 tx=0x7fa200009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f40564c0 con 0x7f61fc0b0b50 2023-07-06T20:14:11.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7f620159b700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61e006adb0 0x7f61e006d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f61ec007fd0 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7f965e7fc700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c2280 0x7f96600c37b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f965800eea0 tx=0x7f965800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0052d0 con 0x7f49280ce2d0 2023-07-06T20:14:11.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c005430 con 0x7f49280ce2d0 2023-07-06T20:14:11.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c003880 con 0x7f49280ce2d0 2023-07-06T20:14:11.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4928163270 con 0x7f49280ce2d0 2023-07-06T20:14:11.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4928163830 con 0x7f49280ce2d0 2023-07-06T20:14:11.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.849+0000 7fbcfbbff700 1 Processor -- start 2023-07-06T20:14:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7fbcfbbff700 1 -- start start 2023-07-06T20:14:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fbcfbbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcf40d20f0 0x7fbcf4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fbcfbbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d2dd0 0x7fbcf40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fbcfbbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcf415b5a0 0x7fbcf40b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fbcfbbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf415a340 con 0x7fbcf40d20f0 2023-07-06T20:14:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fbcfbbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40c76f0 con 0x7fbcf415b5a0 2023-07-06T20:14:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fbcfbbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40c79d0 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658010040 con 0x7f96600c2280 2023-07-06T20:14:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965800ba70 con 0x7f96600c2280 2023-07-06T20:14:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.850+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800bde0 con 0x7f96600c2280 2023-07-06T20:14:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96600c3290 con 0x7f96600c2280 2023-07-06T20:14:11.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966015a550 con 0x7f96600c2280 2023-07-06T20:14:11.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7f6201d9c700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7f61e00735e0 0x7f61e00759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7fbcf999b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcf40d20f0 0x7fbcf4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7fbcf999b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcf40d20f0 0x7fbcf4159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59434/0 (socket says 172.21.15.27:59434) 2023-07-06T20:14:11.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7fbcf999b700 1 -- 172.21.15.27:0/2769421465 learned_addr learned my addr 172.21.15.27:0/2769421465 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:11.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7fbcfa19c700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcf415b5a0 0x7fbcf40b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.852+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49280ca1f0 con 0x7f49280ce2d0 2023-07-06T20:14:11.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.856+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 --> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6dd0002d40 con 0x7f6ddc073660 2023-07-06T20:14:11.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.854+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== osd.3 v2:172.21.15.27:6809/2399591703 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f61e00760a0 con 0x7f61e00735e0 2023-07-06T20:14:11.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.854+0000 7fbcf919a700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d2dd0 0x7fbcf40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.855+0000 7fbcf919a700 1 -- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcf415b5a0 msgr2=0x7fbcf40b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.855+0000 7fbcf919a700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcf415b5a0 0x7fbcf40b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.855+0000 7fbcf919a700 1 -- 172.21.15.27:0/2769421465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcf40d20f0 msgr2=0x7fbcf4159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.855+0000 7fbcf919a700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcf40d20f0 0x7fbcf4159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.855+0000 7fbcf919a700 1 -- 172.21.15.27:0/2769421465 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40094a0 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.855+0000 7fbcf919a700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d2dd0 0x7fbcf40cc6b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c520 tx=0x7fbcf000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.856+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0004c80 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.856+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf0004de0 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.856+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0007310 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.851+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdef8019070 con 0x7fdf080b0440 2023-07-06T20:14:11.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.853+0000 7fdf04ff9700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdee806ad80 0x7fdee806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.854+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdef808a900 con 0x7fdf080b0440 2023-07-06T20:14:11.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.854+0000 7fdf04ff9700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7fdee8073630 0x7fdee8075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.854+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 --> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdee80760f0 con 0x7fdee8073630 2023-07-06T20:14:11.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.854+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef808acb0 con 0x7fdf080b0440 2023-07-06T20:14:11.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.856+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf40c7cb0 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.857+0000 7f6dd77fe700 1 -- 172.21.15.27:0/3237919170 <== osd.1 v2:172.21.15.27:6824/3450674250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6dd0002d40 con 0x7f6ddc073660 2023-07-06T20:14:11.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.857+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf40c8270 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.857+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7f6ddc073660 msgr2=0x7f6ddc075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.857+0000 7f6df57ff700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7f6ddc073660 0x7f6ddc075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbcdc000fc0 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6ddc06adb0 msgr2=0x7f6ddc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f6df57ff700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6ddc06adb0 0x7f6ddc06d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6df014ae30 tx=0x7f6de0009290 comp rx=0 tx=0).stop 2023-07-06T20:14:11.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b0440 msgr2=0x7f6df01695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f6df57ff700 1 --2- 172.21.15.27:0/3237919170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b0440 0x7f6df01695d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6de800bd50 tx=0x7f6de800bd80 comp rx=0 tx=0).stop 2023-07-06T20:14:11.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f6def7fe700 1 -- 172.21.15.27:0/3237919170 reap_dead start 2023-07-06T20:14:11.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.857+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f965801d400 con 0x7f96600c2280 2023-07-06T20:14:11.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.857+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96600c2b60 con 0x7f96600c2280 2023-07-06T20:14:11.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f96477fe700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f964c06adb0 0x7f964c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9658089ba0 con 0x7f96600c2280 2023-07-06T20:14:11.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f96477fe700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f964c0735e0 0x7f964c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 --> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f964c0760a0 con 0x7f964c0735e0 2023-07-06T20:14:11.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 shutdown_connections 2023-07-06T20:14:11.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 >> 172.21.15.27:0/3237919170 conn(0x7f6df00cb440 msgr2=0x7f6df0071300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.856+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f491c0039e0 con 0x7f49280ce2d0 2023-07-06T20:14:11.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f490ffff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f491406ad30 0x7f491406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f491c088b90 con 0x7f49280ce2d0 2023-07-06T20:14:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f490ffff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f49140735e0 0x7f49140759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 --> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49140760a0 con 0x7f49140735e0 2023-07-06T20:14:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.858+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f491c088f00 con 0x7f49280ce2d0 2023-07-06T20:14:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 shutdown_connections 2023-07-06T20:14:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f6df57ff700 1 -- 172.21.15.27:0/3237919170 wait complete. 2023-07-06T20:14:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f965f7fe700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f964c0735e0 0x7f964c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f49277fe700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f491406ad30 0x7f491406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7f4926ffd700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f49140735e0 0x7f49140759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7fdf07fff700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdee806ad80 0x7fdee806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.859+0000 7fdf0cd9c700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7fdee8073630 0x7fdee8075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.860+0000 7f965effd700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f964c06adb0 0x7f964c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.860+0000 7f49277fe700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f491406ad30 0x7f491406d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f492001a040 tx=0x7f492000d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.860+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcf0007470 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.860+0000 7f4926ffd700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f49140735e0 0x7f49140759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.861+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== osd.6 v2:172.21.15.153:6824/2351180975 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f49140760a0 con 0x7f49140735e0 2023-07-06T20:14:11.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7fdf0cd9c700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7fdee8073630 0x7fdee8075a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 --> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a7c164fb0 con 0x7f6a68073660 2023-07-06T20:14:11.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7fbcea7fc700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbcd806ad30 0x7fbcd806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcf0089090 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7fbcea7fc700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fbcd80735e0 0x7fbcd80759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:11.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 --> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbcd80760a0 con 0x7fbcd80735e0 2023-07-06T20:14:11.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf008b670 con 0x7fbcf40d2dd0 2023-07-06T20:14:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.860+0000 7f965effd700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f964c06adb0 0x7f964c06d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9654000c00 tx=0x7f9654007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.861+0000 7f965f7fe700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f964c0735e0 0x7f964c0759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.862+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== osd.5 v2:172.21.15.153:6806/1975714118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f964c0760a0 con 0x7f964c0735e0 2023-07-06T20:14:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.863+0000 7fdf07fff700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdee806ad80 0x7fdee806d230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf080c5300 tx=0x7fdef0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.863+0000 7f6a78ff9700 1 -- 172.21.15.27:0/3170676405 <== osd.0 v2:172.21.15.27:6800/323630915 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a740093d0 con 0x7f6a68073660 2023-07-06T20:14:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.864+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== osd.7 v2:172.21.15.153:6816/469437508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdee80760f0 con 0x7fdee8073630 2023-07-06T20:14:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.863+0000 7fbcf999b700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbcd806ad30 0x7fbcd806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.863+0000 7fbcfa19c700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fbcd80735e0 0x7fbcd80759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.864+0000 7fbcf999b700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbcd806ad30 0x7fbcd806d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00f3b0 tx=0x7fbcec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.864+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f6a68073660 msgr2=0x7f6a68075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.864+0000 7f6a829ff700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f6a68073660 0x7f6a68075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.865+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6806adb0 msgr2=0x7f6a6806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.865+0000 7f6a829ff700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6806adb0 0x7f6a6806d260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cf7a0 tx=0x7f6a6c009040 comp rx=0 tx=0).stop 2023-07-06T20:14:11.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.865+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c0b0440 msgr2=0x7f6a7c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.865+0000 7f6a829ff700 1 --2- 172.21.15.27:0/3170676405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c0b0440 0x7f6a7c0ce550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000e810 tx=0x7f6a7000eb20 comp rx=0 tx=0).stop 2023-07-06T20:14:11.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.865+0000 7f6a80f9c700 1 -- 172.21.15.27:0/3170676405 reap_dead start 2023-07-06T20:14:11.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.866+0000 7fbcfa19c700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fbcd80735e0 0x7fbcd80759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:11.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.866+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 shutdown_connections 2023-07-06T20:14:11.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.866+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 >> 172.21.15.27:0/3170676405 conn(0x7f6a7c0cb440 msgr2=0x7f6a7c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.867+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== osd.4 v2:172.21.15.153:6800/2660030028 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbcd80760a0 con 0x7fbcd80735e0 2023-07-06T20:14:11.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.867+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9658054ce0 con 0x7f96600c2280 2023-07-06T20:14:11.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.867+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 shutdown_connections 2023-07-06T20:14:11.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.867+0000 7f6a829ff700 1 -- 172.21.15.27:0/3170676405 wait complete. 2023-07-06T20:14:11.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.885+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 --> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61fc0b2420 con 0x7f61e00735e0 2023-07-06T20:14:11.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.885+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 --> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa1f0002d40 con 0x7fa1f40735e0 2023-07-06T20:14:11.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.886+0000 7fa205ffb700 1 -- 172.21.15.27:0/1554373322 <== osd.2 v2:172.21.15.27:6808/380505300 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa1f0002d40 con 0x7fa1f40735e0 2023-07-06T20:14:11.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.887+0000 7f61f1ffb700 1 -- 172.21.15.27:0/3142766138 <== osd.3 v2:172.21.15.27:6809/2399591703 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61fc0b2420 con 0x7f61e00735e0 2023-07-06T20:14:11.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.890+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7fa1f40735e0 msgr2=0x7fa1f40759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.890+0000 7fa2179ff700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7fa1f40735e0 0x7fa1f40759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.890+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7f61e00735e0 msgr2=0x7f61e00759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.890+0000 7f62037ff700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7f61e00735e0 0x7f61e00759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.890+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61e006adb0 msgr2=0x7f61e006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7f62037ff700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61e006adb0 0x7f61e006d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f61ec007fd0 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2023-07-06T20:14:11.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0b0b50 msgr2=0x7f61fc0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7f62037ff700 1 --2- 172.21.15.27:0/3142766138 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61fc0b0b50 0x7f61fc0b30a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f61fc06f2d0 tx=0x7f61f4004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7f6201d9c700 1 -- 172.21.15.27:0/3142766138 reap_dead start 2023-07-06T20:14:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa1f406adb0 msgr2=0x7fa1f406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa2179ff700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa1f406adb0 0x7fa1f406d260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa210169610 tx=0x7fa200009040 comp rx=0 tx=0).stop 2023-07-06T20:14:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2100cbf50 msgr2=0x7fa2100c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa2179ff700 1 --2- 172.21.15.27:0/1554373322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2100cbf50 0x7fa2100c6700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b970 tx=0x7fa20800bd30 comp rx=0 tx=0).stop 2023-07-06T20:14:11.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa215f9c700 1 -- 172.21.15.27:0/1554373322 reap_dead start 2023-07-06T20:14:11.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 shutdown_connections 2023-07-06T20:14:11.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 >> 172.21.15.27:0/1554373322 conn(0x7fa2100cb440 msgr2=0x7fa21014d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 shutdown_connections 2023-07-06T20:14:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.891+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 >> 172.21.15.27:0/3142766138 conn(0x7f61fc0cb440 msgr2=0x7f61fc0ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.892+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 shutdown_connections 2023-07-06T20:14:11.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.892+0000 7fa2179ff700 1 -- 172.21.15.27:0/1554373322 wait complete. 2023-07-06T20:14:11.923 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2023-07-06T20:14:11.923 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T20:14:11.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.893+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 shutdown_connections 2023-07-06T20:14:11.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.893+0000 7f62037ff700 1 -- 172.21.15.27:0/3142766138 wait complete. 2023-07-06T20:14:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.896+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 --> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f492816dba0 con 0x7f49140735e0 2023-07-06T20:14:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f490ffff700 1 -- 172.21.15.27:0/4042922241 <== osd.6 v2:172.21.15.153:6824/2351180975 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4918009b80 con 0x7f49140735e0 2023-07-06T20:14:11.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f49140735e0 msgr2=0x7f49140759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f492e1ff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f49140735e0 0x7f49140759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f491406ad30 msgr2=0x7f491406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f492e1ff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f491406ad30 0x7f491406d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f492001a040 tx=0x7f492000d040 comp rx=0 tx=0).stop 2023-07-06T20:14:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49280ce2d0 msgr2=0x7f4928161110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7f492e1ff700 1 --2- 172.21.15.27:0/4042922241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49280ce2d0 0x7f4928161110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c390 tx=0x7f491c00c6a0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.897+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 --> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf0806b4e0 con 0x7fdee8073630 2023-07-06T20:14:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 --> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96600ca050 con 0x7f964c0735e0 2023-07-06T20:14:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7f4927fff700 1 -- 172.21.15.27:0/4042922241 reap_dead start 2023-07-06T20:14:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 shutdown_connections 2023-07-06T20:14:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 >> 172.21.15.27:0/4042922241 conn(0x7f49280cb440 msgr2=0x7f492806b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.928 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:11.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T20:14:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 shutdown_connections 2023-07-06T20:14:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7f492e1ff700 1 -- 172.21.15.27:0/4042922241 wait complete. 2023-07-06T20:14:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.899+0000 7f96477fe700 1 -- 172.21.15.27:0/2236871233 <== osd.5 v2:172.21.15.153:6806/1975714118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96500013d0 con 0x7f964c0735e0 2023-07-06T20:14:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.898+0000 7fdf04ff9700 1 -- 172.21.15.27:0/1231000235 <== osd.7 v2:172.21.15.153:6816/469437508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf00009280 con 0x7fdee8073630 2023-07-06T20:14:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.899+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f964c0735e0 msgr2=0x7f964c0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.899+0000 7f96657ff700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f964c0735e0 0x7f964c0759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f964c06adb0 msgr2=0x7f964c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f96657ff700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f964c06adb0 0x7f964c06d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9654000c00 tx=0x7f9654007000 comp rx=0 tx=0).stop 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c2280 msgr2=0x7f96600c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f96657ff700 1 --2- 172.21.15.27:0/2236871233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600c2280 0x7f96600c37b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f965800eea0 tx=0x7f965800b390 comp rx=0 tx=0).stop 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f965f7fe700 1 -- 172.21.15.27:0/2236871233 reap_dead start 2023-07-06T20:14:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 shutdown_connections 2023-07-06T20:14:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 >> 172.21.15.27:0/2236871233 conn(0x7f96600cb440 msgr2=0x7f96600ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7fdee8073630 msgr2=0x7fdee8075a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7fdf0e7ff700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7fdee8073630 0x7fdee8075a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdee806ad80 msgr2=0x7fdee806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7fdf0e7ff700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdee806ad80 0x7fdee806d230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf080c5300 tx=0x7fdef0008040 comp rx=0 tx=0).stop 2023-07-06T20:14:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 msgr2=0x7fdf080aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.900+0000 7fdf0e7ff700 1 --2- 172.21.15.27:0/1231000235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf080b0440 0x7fdf080aeb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf0806f2d0 tx=0x7fdef8004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.901+0000 7fdf0cd9c700 1 -- 172.21.15.27:0/1231000235 reap_dead start 2023-07-06T20:14:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.901+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 shutdown_connections 2023-07-06T20:14:11.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.901+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 >> 172.21.15.27:0/1231000235 conn(0x7fdf080cb440 msgr2=0x7fdf080af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.901+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 shutdown_connections 2023-07-06T20:14:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.901+0000 7f96657ff700 1 -- 172.21.15.27:0/2236871233 wait complete. 2023-07-06T20:14:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.901+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 --> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbcdc002d40 con 0x7fbcd80735e0 2023-07-06T20:14:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcea7fc700 1 -- 172.21.15.27:0/2769421465 <== osd.4 v2:172.21.15.153:6800/2660030028 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbcdc002d40 con 0x7fbcd80735e0 2023-07-06T20:14:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fbcd80735e0 msgr2=0x7fbcd80759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfbbff700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fbcd80735e0 0x7fbcd80759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbcd806ad30 msgr2=0x7fbcd806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfbbff700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbcd806ad30 0x7fbcd806d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00f3b0 tx=0x7fbcec009040 comp rx=0 tx=0).stop 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d2dd0 msgr2=0x7fbcf40cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfbbff700 1 --2- 172.21.15.27:0/2769421465 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcf40d2dd0 0x7fbcf40cc6b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c520 tx=0x7fbcf000c830 comp rx=0 tx=0).stop 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.907+0000 7fbcfa19c700 1 -- 172.21.15.27:0/2769421465 reap_dead start 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.908+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 shutdown_connections 2023-07-06T20:14:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.908+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 >> 172.21.15.27:0/2769421465 conn(0x7fbcf406c150 msgr2=0x7fbcf415ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.908+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 shutdown_connections 2023-07-06T20:14:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.908+0000 7fbcfbbff700 1 -- 172.21.15.27:0/2769421465 wait complete. 2023-07-06T20:14:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.913+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 shutdown_connections 2023-07-06T20:14:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:11.913+0000 7fdf0e7ff700 1 -- 172.21.15.27:0/1231000235 wait complete. 2023-07-06T20:14:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2023-07-06T20:14:11.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T20:14:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:55834574858 2023-07-06T20:14:11.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T20:14:11.940 INFO:teuthology.orchestra.run.smithi027.stdout:55834574858 2023-07-06T20:14:11.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T20:14:11.942 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:11.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T20:14:11.943 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:11.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T20:14:11.944 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2023-07-06T20:14:11.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T20:14:12.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.254+0000 7f7b9d1ff700 1 Processor -- start 2023-07-06T20:14:12.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.254+0000 7f7b9d1ff700 1 -- start start 2023-07-06T20:14:12.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.257+0000 7f7b9d1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98165290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.257+0000 7f7b9d1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7b980ad8f0 0x7f7b980add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.257+0000 7f7b9d1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980cdec0 con 0x7f7b980c1870 2023-07-06T20:14:12.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.257+0000 7f7b9d1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980ae2d0 con 0x7f7b980ad8f0 2023-07-06T20:14:12.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.257+0000 7f7b9d1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980ce120 con 0x7f7b98160bc0 2023-07-06T20:14:12.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.259+0000 7f7b9759e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7b980ad8f0 0x7f7b980add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.260+0000 7f7b96d9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.260+0000 7f7b9659c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7b980c1870 0x7f7b980cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48690/0 (socket says 172.21.15.27:48690) 2023-07-06T20:14:12.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.260+0000 7f7b9659c700 1 -- 172.21.15.27:0/3483614032 learned_addr learned my addr 172.21.15.27:0/3483614032 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.261+0000 7f7b9759e700 1 -- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 msgr2=0x7f7b98165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.261+0000 7f7b9759e700 1 --2- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.261+0000 7f7b9759e700 1 -- 172.21.15.27:0/3483614032 >> v1:172.21.15.153:6789/0 conn(0x7f7b980c1870 legacy=0x7f7b980cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.261+0000 7f7b9759e700 1 -- 172.21.15.27:0/3483614032 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b981647b0 con 0x7f7b980ad8f0 2023-07-06T20:14:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.261+0000 7f7b9759e700 1 --2- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980ad8f0 0x7f7b980add90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009d90 tx=0x7f7b9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df21b826c12185e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.262+0000 7f7b95d9b700 1 -- 172.21.15.27:0/3483614032 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000e070 con 0x7f7b980ad8f0 2023-07-06T20:14:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.262+0000 7f7b95d9b700 1 -- 172.21.15.27:0/3483614032 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000bb20 con 0x7f7b980ad8f0 2023-07-06T20:14:12.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.262+0000 7f7b95d9b700 1 -- 172.21.15.27:0/3483614032 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90012430 con 0x7f7b980ad8f0 2023-07-06T20:14:12.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.263+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980ad8f0 msgr2=0x7f7b980add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.263+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980ad8f0 0x7f7b980add90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009d90 tx=0x7f7b9000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:12.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.264+0000 7f41981ff700 1 Processor -- start 2023-07-06T20:14:12.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.264+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/3483614032 shutdown_connections 2023-07-06T20:14:12.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.264+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.264+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/3483614032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980ad8f0 0x7f7b980add90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.264+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/3483614032 >> 172.21.15.27:0/3483614032 conn(0x7f7b980cb440 msgr2=0x7f7b980cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.265+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/3483614032 shutdown_connections 2023-07-06T20:14:12.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.265+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/3483614032 wait complete. 2023-07-06T20:14:12.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.266+0000 7f41981ff700 1 -- start start 2023-07-06T20:14:12.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.268+0000 7f41981ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41901589e0 0x7f41900b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.268+0000 7f41981ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f41900caeb0 0x7f41900b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.268+0000 7f41981ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419006feb0 con 0x7f4190158610 2023-07-06T20:14:12.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.268+0000 7f41981ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419006c780 con 0x7f41901589e0 2023-07-06T20:14:12.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.268+0000 7f41981ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419006c4a0 con 0x7f41900caeb0 2023-07-06T20:14:12.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.269+0000 7f419579a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41901589e0 0x7f41900b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.269+0000 7f419579a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41901589e0 0x7f41900b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44698/0 (socket says 172.21.15.27:44698) 2023-07-06T20:14:12.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.269+0000 7f419579a700 1 -- 172.21.15.27:0/1276509511 learned_addr learned my addr 172.21.15.27:0/1276509511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.270+0000 7f419579a700 1 -- 172.21.15.27:0/1276509511 >> v2:172.21.15.153:3300/0 conn(0x7f41900caeb0 msgr2=0x7f41900b0980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.270+0000 7f419579a700 1 --2- 172.21.15.27:0/1276509511 >> v2:172.21.15.153:3300/0 conn(0x7f41900caeb0 0x7f41900b0980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.270+0000 7f419579a700 1 -- 172.21.15.27:0/1276509511 >> v1:172.21.15.153:6789/0 conn(0x7f4190158610 legacy=0x7f419006bc30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.270+0000 7f419579a700 1 -- 172.21.15.27:0/1276509511 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419006f860 con 0x7f41901589e0 2023-07-06T20:14:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.270+0000 7f419579a700 1 --2- 172.21.15.27:0/1276509511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 0x7f41900b0440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41900ca9d0 tx=0x7f418c00d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a18c300b8aace server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.270+0000 7f7b9d1ff700 1 Processor -- start 2023-07-06T20:14:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.271+0000 7f4194f99700 1 -- 172.21.15.27:0/1276509511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c010070 con 0x7f41901589e0 2023-07-06T20:14:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.271+0000 7f4194f99700 1 -- 172.21.15.27:0/1276509511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c00dca0 con 0x7f41901589e0 2023-07-06T20:14:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.271+0000 7f4194f99700 1 -- 172.21.15.27:0/1276509511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c007d70 con 0x7f41901589e0 2023-07-06T20:14:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f41981ff700 1 -- 172.21.15.27:0/1276509511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 msgr2=0x7f41900b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f41981ff700 1 --2- 172.21.15.27:0/1276509511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 0x7f41900b0440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41900ca9d0 tx=0x7f418c00d650 comp rx=0 tx=0).stop 2023-07-06T20:14:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.271+0000 7f7b9d1ff700 1 -- start start 2023-07-06T20:14:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f7b9d1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980ad8f0 0x7f7b980b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f7b9d1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980c1870 0x7f7b98153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f7b9d1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98161920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f7b9d1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980b3380 con 0x7f7b980ad8f0 2023-07-06T20:14:12.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f7b9d1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981611f0 con 0x7f7b980c1870 2023-07-06T20:14:12.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.272+0000 7f7b9d1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980b2ae0 con 0x7f7b98160bc0 2023-07-06T20:14:12.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.273+0000 7f7b9759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.273+0000 7f7b9759e700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98161920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:47706/0 (socket says 172.21.15.27:47706) 2023-07-06T20:14:12.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.273+0000 7f7b9759e700 1 -- 172.21.15.27:0/1542792974 learned_addr learned my addr 172.21.15.27:0/1542792974 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.274+0000 7f7b9759e700 1 -- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980c1870 msgr2=0x7f7b98153350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:14:12.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.274+0000 7f7b9759e700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980c1870 0x7f7b98153350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.274+0000 7f7b9759e700 1 -- 172.21.15.27:0/1542792974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980ad8f0 msgr2=0x7f7b980b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.274+0000 7f7b9759e700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980ad8f0 0x7f7b980b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.274+0000 7f7b9759e700 1 -- 172.21.15.27:0/1542792974 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009a70 con 0x7f7b98160bc0 2023-07-06T20:14:12.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.274+0000 7f7b9759e700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98161920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000b370 tx=0x7f7b90004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f41981ff700 1 -- 172.21.15.27:0/1276509511 shutdown_connections 2023-07-06T20:14:12.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f41981ff700 1 --2- 172.21.15.27:0/1276509511 >> v2:172.21.15.153:3300/0 conn(0x7f41900caeb0 0x7f41900b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f41981ff700 1 --2- 172.21.15.27:0/1276509511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 0x7f41900b0440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f41981ff700 1 -- 172.21.15.27:0/1276509511 >> 172.21.15.27:0/1276509511 conn(0x7f41900cb440 msgr2=0x7f41900cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900048e0 con 0x7f7b98160bc0 2023-07-06T20:14:12.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90020070 con 0x7f7b98160bc0 2023-07-06T20:14:12.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900276c0 con 0x7f7b98160bc0 2023-07-06T20:14:12.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.275+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980b2dc0 con 0x7f7b98160bc0 2023-07-06T20:14:12.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.276+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b98161400 con 0x7f7b98160bc0 2023-07-06T20:14:12.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.278+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980b4770 con 0x7f7b98160bc0 2023-07-06T20:14:12.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.278+0000 7f41981ff700 1 -- 172.21.15.27:0/1276509511 shutdown_connections 2023-07-06T20:14:12.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.278+0000 7f41981ff700 1 -- 172.21.15.27:0/1276509511 wait complete. 2023-07-06T20:14:12.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.281+0000 7f41981ff700 1 Processor -- start 2023-07-06T20:14:12.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.280+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b90027820 con 0x7f7b98160bc0 2023-07-06T20:14:12.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.282+0000 7f7b7f7fe700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7b8406adb0 0x7f7b8406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.282+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b90022070 con 0x7f7b98160bc0 2023-07-06T20:14:12.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.283+0000 7f7b96d9d700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7b8406adb0 0x7f7b8406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.283+0000 7f41981ff700 1 -- start start 2023-07-06T20:14:12.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f41981ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41900caeb0 0x7f4190157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f41981ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 0x7f41900c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f41981ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 0x7f41900c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f41981ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900c70f0 con 0x7f41901589e0 2023-07-06T20:14:12.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f41981ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190157640 con 0x7f41900caeb0 2023-07-06T20:14:12.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f41981ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41900c64e0 con 0x7f4190158610 2023-07-06T20:14:12.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f419579a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 0x7f41900c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f419579a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 0x7f41900c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:47720/0 (socket says 172.21.15.27:47720) 2023-07-06T20:14:12.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.285+0000 7f419579a700 1 -- 172.21.15.27:0/117987168 learned_addr learned my addr 172.21.15.27:0/117987168 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.286+0000 7f419579a700 1 -- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41900caeb0 msgr2=0x7f4190157100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:14:12.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.286+0000 7f419579a700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41900caeb0 0x7f4190157100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.286+0000 7f419579a700 1 -- 172.21.15.27:0/117987168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 msgr2=0x7f41900c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.286+0000 7f419579a700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 0x7f41900c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.286+0000 7f419579a700 1 -- 172.21.15.27:0/117987168 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c00d050 con 0x7f4190158610 2023-07-06T20:14:12.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.289+0000 7f7b96d9d700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7b8406adb0 0x7f7b8406d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b88006b20 tx=0x7f7b88006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.289+0000 7f419579a700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 0x7f41900c7580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f418c018040 tx=0x7f418c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.290+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b90056ab0 con 0x7f7b98160bc0 2023-07-06T20:14:12.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.289+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0046d0 con 0x7f4190158610 2023-07-06T20:14:12.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.290+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c004830 con 0x7f4190158610 2023-07-06T20:14:12.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.290+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c008270 con 0x7f4190158610 2023-07-06T20:14:12.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.290+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41900c67c0 con 0x7f4190158610 2023-07-06T20:14:12.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.290+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41900c6d80 con 0x7f4190158610 2023-07-06T20:14:12.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.293+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41900008d0 con 0x7f4190158610 2023-07-06T20:14:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.296+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f418c0083d0 con 0x7f4190158610 2023-07-06T20:14:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.298+0000 7f41867fc700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f417406ad80 0x7f417406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.298+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f418c08c030 con 0x7f4190158610 2023-07-06T20:14:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.300+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f418c057150 con 0x7f4190158610 2023-07-06T20:14:12.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.300+0000 7f4195f9b700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f417406ad80 0x7f417406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.337+0000 7f4195f9b700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f417406ad80 0x7f417406d230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4180009710 tx=0x7f4180006c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.373+0000 7f493d7ff700 1 Processor -- start 2023-07-06T20:14:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.373+0000 7f493d7ff700 1 -- start start 2023-07-06T20:14:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.375+0000 7f493d7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49380d0220 0x7f493806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f493d7ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49380cdbb0 con 0x7f493806ec60 2023-07-06T20:14:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f493d7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49380cd880 con 0x7f4938159620 2023-07-06T20:14:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f493d7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493806f0e0 con 0x7f49380d0220 2023-07-06T20:14:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.374+0000 7f62935ff700 1 Processor -- start 2023-07-06T20:14:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.374+0000 7f62935ff700 1 -- start start 2023-07-06T20:14:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f4936ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4938159620 0x7f49380ca790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48708/0 (socket says 172.21.15.27:48708) 2023-07-06T20:14:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.377+0000 7f4936ffd700 1 -- 172.21.15.27:0/2096561491 learned_addr learned my addr 172.21.15.27:0/2096561491 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.377+0000 7f49367fc700 1 --2- 172.21.15.27:0/2096561491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49380d0220 0x7f493806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f62935ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f628c070db0 0x7f628c0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f62935ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f628c06d800 0x7f628c0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f62935ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c06bb30 con 0x7f628c0cac80 2023-07-06T20:14:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f62935ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c06b220 con 0x7f628c06d800 2023-07-06T20:14:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.376+0000 7f62935ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0cb140 con 0x7f628c070db0 2023-07-06T20:14:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.377+0000 7f49367fc700 1 -- 172.21.15.27:0/2096561491 >> v1:172.21.15.153:6789/0 conn(0x7f4938159620 legacy=0x7f49380ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.377+0000 7f49367fc700 1 -- 172.21.15.27:0/2096561491 >> v1:172.21.15.27:6789/0 conn(0x7f493806ec60 legacy=0x7f49380ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.377+0000 7f49367fc700 1 -- 172.21.15.27:0/2096561491 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380cb5a0 con 0x7f49380d0220 2023-07-06T20:14:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f628c06d800 0x7f628c0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f628c06d800 0x7f628c0b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44744/0 (socket says 172.21.15.27:44744) 2023-07-06T20:14:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 -- 172.21.15.27:0/926239751 learned_addr learned my addr 172.21.15.27:0/926239751 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f49367fc700 1 --2- 172.21.15.27:0/2096561491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49380d0220 0x7f493806e720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f492c010be0 tx=0x7f492c010ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b40836883e5255f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.379+0000 7f4935ffb700 1 -- 172.21.15.27:0/2096561491 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00bea0 con 0x7f49380d0220 2023-07-06T20:14:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.379+0000 7f4935ffb700 1 -- 172.21.15.27:0/2096561491 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c004030 con 0x7f49380d0220 2023-07-06T20:14:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.379+0000 7f4935ffb700 1 -- 172.21.15.27:0/2096561491 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0043a0 con 0x7f49380d0220 2023-07-06T20:14:12.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 -- 172.21.15.27:0/926239751 >> v2:172.21.15.153:3300/0 conn(0x7f628c070db0 msgr2=0x7f628c0b0440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:12.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 --2- 172.21.15.27:0/926239751 >> v2:172.21.15.153:3300/0 conn(0x7f628c070db0 0x7f628c0b0440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 -- 172.21.15.27:0/926239751 >> v1:172.21.15.27:6789/0 conn(0x7f628c0cac80 legacy=0x7f628c0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.378+0000 7f6291b9c700 1 -- 172.21.15.27:0/926239751 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c06b4e0 con 0x7f628c06d800 2023-07-06T20:14:12.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.379+0000 7f6291b9c700 1 --2- 172.21.15.27:0/926239751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0b0980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f628c06ee10 tx=0x7f6280009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef75d95c86699d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 -- 172.21.15.27:0/2096561491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49380d0220 msgr2=0x7f493806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 --2- 172.21.15.27:0/2096561491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49380d0220 0x7f493806e720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f492c010be0 tx=0x7f492c010ef0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 -- 172.21.15.27:0/2096561491 shutdown_connections 2023-07-06T20:14:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 --2- 172.21.15.27:0/2096561491 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f49380d0220 0x7f493806e720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 -- 172.21.15.27:0/2096561491 >> 172.21.15.27:0/2096561491 conn(0x7f49380b22f0 msgr2=0x7f49380cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 -- 172.21.15.27:0/2096561491 shutdown_connections 2023-07-06T20:14:12.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f493d7ff700 1 -- 172.21.15.27:0/2096561491 wait complete. 2023-07-06T20:14:12.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f404a9ff700 1 Processor -- start 2023-07-06T20:14:12.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f404a9ff700 1 -- start start 2023-07-06T20:14:12.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f627ffff700 1 -- 172.21.15.27:0/926239751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280004230 con 0x7f628c06d800 2023-07-06T20:14:12.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f627ffff700 1 -- 172.21.15.27:0/926239751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6280004390 con 0x7f628c06d800 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.380+0000 7f627ffff700 1 -- 172.21.15.27:0/926239751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800036a0 con 0x7f628c06d800 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f62935ff700 1 -- 172.21.15.27:0/926239751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 msgr2=0x7f628c0b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f62935ff700 1 --2- 172.21.15.27:0/926239751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0b0980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f628c06ee10 tx=0x7f6280009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f62935ff700 1 -- 172.21.15.27:0/926239751 shutdown_connections 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f62935ff700 1 --2- 172.21.15.27:0/926239751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0b0980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f62935ff700 1 --2- 172.21.15.27:0/926239751 >> v2:172.21.15.153:3300/0 conn(0x7f628c070db0 0x7f628c0b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.381+0000 7f62935ff700 1 -- 172.21.15.27:0/926239751 >> 172.21.15.27:0/926239751 conn(0x7f628c0cb440 msgr2=0x7f628c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.383+0000 7f493d7ff700 1 Processor -- start 2023-07-06T20:14:12.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.383+0000 7f493d7ff700 1 -- start start 2023-07-06T20:14:12.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.385+0000 7f493d7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 0x7f4938155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f404a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40440cbf50 0x7f404406bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f404a9ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40440caeb0 0x7f4044158610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f404a9ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404406feb0 con 0x7f40440b0440 2023-07-06T20:14:12.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f404a9ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404406c780 con 0x7f40440caeb0 2023-07-06T20:14:12.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f404a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404415f670 con 0x7f40440cbf50 2023-07-06T20:14:12.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.385+0000 7f4048f9c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f40440b0440 0x7f404415c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36066/0 (socket says 172.21.15.27:36066) 2023-07-06T20:14:12.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.385+0000 7f4048f9c700 1 -- 172.21.15.27:0/2624938444 learned_addr learned my addr 172.21.15.27:0/2624938444 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:12.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f493d7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49380d0220 0x7f493815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f493d7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4938159620 0x7f49380ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f493d7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49380c3190 con 0x7f49380d0220 2023-07-06T20:14:12.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f493d7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493815ca30 con 0x7f4938159620 2023-07-06T20:14:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f493d7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493815b720 con 0x7f493806ec60 2023-07-06T20:14:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f4936ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 0x7f4938155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f4936ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 0x7f4938155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:47742/0 (socket says 172.21.15.27:47742) 2023-07-06T20:14:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f4936ffd700 1 -- 172.21.15.27:0/3477886289 learned_addr learned my addr 172.21.15.27:0/3477886289 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082589576 0 0) 0x7f404406feb0 con 0x7f40440b0440 2023-07-06T20:14:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f402c0035b0 con 0x7f40440b0440 2023-07-06T20:14:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f49367fc700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49380d0220 0x7f493815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4936ffd700 1 -- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4938159620 msgr2=0x7f49380ce850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:14:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4936ffd700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4938159620 0x7f49380ce850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4936ffd700 1 -- 172.21.15.27:0/3477886289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49380d0220 msgr2=0x7f493815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4936ffd700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49380d0220 0x7f493815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4936ffd700 1 -- 172.21.15.27:0/3477886289 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c010810 con 0x7f493806ec60 2023-07-06T20:14:12.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f49367fc700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49380d0220 0x7f493815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174881575 0 0) 0x7f402c0035b0 con 0x7f40440b0440 2023-07-06T20:14:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40440cbf50 msgr2=0x7f404406bc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:14:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f40440cbf50 0x7f404406bc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 >> v2:172.21.15.153:3300/0 conn(0x7f40440caeb0 msgr2=0x7f4044158610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 --2- 172.21.15.27:0/2624938444 >> v2:172.21.15.153:3300/0 conn(0x7f40440caeb0 0x7f4044158610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404406f860 con 0x7f40440b0440 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f4038001670 con 0x7f40440b0440 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 >> v1:172.21.15.27:6789/0 conn(0x7f40440b0440 legacy=0x7f404415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c003a00 0x7f402c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f402c004850 0x7f402c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f402c005320 0x7f402c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404406c780 con 0x7f402c003a00 2023-07-06T20:14:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404415f670 con 0x7f402c004850 2023-07-06T20:14:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c00c1a0 con 0x7f402c005320 2023-07-06T20:14:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f4936ffd700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 0x7f4938155fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f492800b550 tx=0x7f492800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.382+0000 7f5a6e3ff700 1 Processor -- start 2023-07-06T20:14:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.382+0000 7f5a6e3ff700 1 -- start start 2023-07-06T20:14:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f5a6e3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a68160bc0 0x7f5a68165290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f5a6e3ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a680c1870 0x7f5a680cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f5a6e3ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680ce120 con 0x7f5a680ad8f0 2023-07-06T20:14:12.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f5a6e3ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a681647b0 con 0x7f5a680c1870 2023-07-06T20:14:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.384+0000 7f5a6e3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68164a90 con 0x7f5a68160bc0 2023-07-06T20:14:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.385+0000 7f5a5f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a680c1870 0x7f5a680cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.385+0000 7f5a6c99c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5a680ad8f0 0x7f5a680cc590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36056/0 (socket says 172.21.15.27:36056) 2023-07-06T20:14:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.385+0000 7f5a6c99c700 1 -- 172.21.15.27:0/3540166241 learned_addr learned my addr 172.21.15.27:0/3540166241 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f5a677fe700 1 -- 172.21.15.27:0/3540166241 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435186669 0 0) 0x7f5a680ce120 con 0x7f5a680ad8f0 2023-07-06T20:14:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f62935ff700 1 -- 172.21.15.27:0/926239751 shutdown_connections 2023-07-06T20:14:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.386+0000 7f5a677fe700 1 -- 172.21.15.27:0/3540166241 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a540035b0 con 0x7f5a680ad8f0 2023-07-06T20:14:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f5a5f7fe700 1 -- 172.21.15.27:0/3540166241 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a68160bc0 msgr2=0x7f5a68165290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f5a5f7fe700 1 --2- 172.21.15.27:0/3540166241 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a68160bc0 0x7f5a68165290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f5a5f7fe700 1 -- 172.21.15.27:0/3540166241 >> v1:172.21.15.27:6789/0 conn(0x7f5a680ad8f0 legacy=0x7f5a680cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f5a5f7fe700 1 -- 172.21.15.27:0/3540166241 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68164d70 con 0x7f5a680c1870 2023-07-06T20:14:12.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.387+0000 7f5a5f7fe700 1 --2- 172.21.15.27:0/3540166241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a680c1870 0x7f5a680cd7b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009840 tx=0x7f5a50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e95937fe1b99136 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f5a677fe700 1 -- 172.21.15.27:0/3540166241 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50011830 con 0x7f5a680c1870 2023-07-06T20:14:12.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f5a677fe700 1 -- 172.21.15.27:0/3540166241 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a50011990 con 0x7f5a680c1870 2023-07-06T20:14:12.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f5a677fe700 1 -- 172.21.15.27:0/3540166241 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50011d20 con 0x7f5a680c1870 2023-07-06T20:14:12.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f4048f9c700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f402c005320 0x7f402c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f4048f9c700 1 -- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f402c004850 msgr2=0x7f402c004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:14:12.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f4048f9c700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f402c004850 0x7f402c004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f4048f9c700 1 -- 172.21.15.27:0/2624938444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c003a00 msgr2=0x7f402c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f4048f9c700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c003a00 0x7f402c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f4048f9c700 1 -- 172.21.15.27:0/2624938444 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404406f860 con 0x7f402c005320 2023-07-06T20:14:12.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f4ab47ff700 1 Processor -- start 2023-07-06T20:14:12.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.388+0000 7f62935ff700 1 -- 172.21.15.27:0/926239751 wait complete. 2023-07-06T20:14:12.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/3540166241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a680c1870 msgr2=0x7f5a680cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.389+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/3540166241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a680c1870 0x7f5a680cd7b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009840 tx=0x7f5a50009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:12.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f4048f9c700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f402c005320 0x7f402c00ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3f0 tx=0x7f402c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89579ea309839c8a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928011830 con 0x7f493806ec60 2023-07-06T20:14:12.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928011990 con 0x7f493806ec60 2023-07-06T20:14:12.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928011d40 con 0x7f493806ec60 2023-07-06T20:14:12.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493815b9d0 con 0x7f493806ec60 2023-07-06T20:14:12.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038005960 con 0x7f402c005320 2023-07-06T20:14:12.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038005ac0 con 0x7f402c005320 2023-07-06T20:14:12.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f4042ffd700 1 -- 172.21.15.27:0/2624938444 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038005e70 con 0x7f402c005320 2023-07-06T20:14:12.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/3540166241 shutdown_connections 2023-07-06T20:14:12.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/3540166241 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a68160bc0 0x7f5a68165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/3540166241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a680c1870 0x7f5a680cd7b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/3540166241 >> 172.21.15.27:0/3540166241 conn(0x7f5a680cb440 msgr2=0x7f5a680cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f7ae8dff700 1 Processor -- start 2023-07-06T20:14:12.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f7ae8dff700 1 -- start start 2023-07-06T20:14:12.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f404a9ff700 1 -- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f402c005320 msgr2=0x7f402c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2624938444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f402c005320 0x7f402c00ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3f0 tx=0x7f402c005880 comp rx=0 tx=0).stop 2023-07-06T20:14:12.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f4048f9c700 1 -- 172.21.15.27:0/2624938444 reap_dead start 2023-07-06T20:14:12.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.392+0000 7f404a9ff700 1 -- 172.21.15.27:0/2624938444 shutdown_connections 2023-07-06T20:14:12.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.392+0000 7f404a9ff700 1 -- 172.21.15.27:0/2624938444 >> 172.21.15.27:0/2624938444 conn(0x7f40440cb440 msgr2=0x7f40440cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f4ab47ff700 1 -- start start 2023-07-06T20:14:12.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.392+0000 7f4ab47ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4aac0b0b50 0x7f4aac163690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.392+0000 7f4ab47ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac06feb0 con 0x7f4aac158610 2023-07-06T20:14:12.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.392+0000 7f4ab47ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac06c780 con 0x7f4aac1589e0 2023-07-06T20:14:12.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.392+0000 7f4ab47ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac06c4a0 con 0x7f4aac0b0b50 2023-07-06T20:14:12.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/3540166241 shutdown_connections 2023-07-06T20:14:12.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f62935ff700 1 Processor -- start 2023-07-06T20:14:12.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.391+0000 7f62935ff700 1 -- start start 2023-07-06T20:14:12.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/3540166241 wait complete. 2023-07-06T20:14:12.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.390+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49380c36e0 con 0x7f493806ec60 2023-07-06T20:14:12.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f4ab2d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4aac0b0b50 0x7f4aac163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f4ab2d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4aac0b0b50 0x7f4aac163690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44784/0 (socket says 172.21.15.27:44784) 2023-07-06T20:14:12.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f4ab2d9c700 1 -- 172.21.15.27:0/3412342634 learned_addr learned my addr 172.21.15.27:0/3412342634 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49180052f0 con 0x7f493806ec60 2023-07-06T20:14:12.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae8dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 0x7f7ae40ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae8dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ae40b2180 0x7f7ae40cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae8dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ae40cfcb0 0x7f7ae4160130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae8dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40cda40 con 0x7f7ae40cfcb0 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae8dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40d0130 con 0x7f7ae40b2180 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae8dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40cd480 con 0x7f7ae40b8660 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae259c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 0x7f7ae40ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae259c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 0x7f7ae40ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:47760/0 (socket says 172.21.15.27:47760) 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f7ae259c700 1 -- 172.21.15.27:0/600464419 learned_addr learned my addr 172.21.15.27:0/600464419 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae2d9d700 1 --2- 172.21.15.27:0/600464419 >> v2:172.21.15.27:3300/0 conn(0x7f7ae40cfcb0 0x7f7ae4160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae1d9b700 1 --2- 172.21.15.27:0/600464419 >> v2:172.21.15.153:3300/0 conn(0x7f7ae40b2180 0x7f7ae40cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f4ab2d9c700 1 -- 172.21.15.27:0/3412342634 >> v1:172.21.15.153:6789/0 conn(0x7f4aac1589e0 legacy=0x7f4aac0b0440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:14:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f4ab2d9c700 1 -- 172.21.15.27:0/3412342634 >> v1:172.21.15.27:6789/0 conn(0x7f4aac158610 legacy=0x7f4aac06bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f4ab2d9c700 1 -- 172.21.15.27:0/3412342634 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac06f860 con 0x7f4aac0b0b50 2023-07-06T20:14:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f4ab2d9c700 1 --2- 172.21.15.27:0/3412342634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac0b0b50 0x7f4aac163690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400d370 tx=0x7f4aa400d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc7aabf7e18bf9f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f5a6e3ff700 1 Processor -- start 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f5a6e3ff700 1 -- start start 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f5a6e3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a680ad8f0 0x7f5a68151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f5a6e3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a680c1870 0x7f5a680b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f5a6e3ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a68160bc0 0x7f5a681540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f5a6e3ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6814f290 con 0x7f5a68160bc0 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f5a6e3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6814e680 con 0x7f5a680ad8f0 2023-07-06T20:14:12.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f5a6e3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6814e960 con 0x7f5a680c1870 2023-07-06T20:14:12.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f62935ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f62935ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 0x7f628c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f62935ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f628c0cac80 0x7f628c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f62935ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c64a0 con 0x7f628c06d800 2023-07-06T20:14:12.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f62935ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c6780 con 0x7f628c070db0 2023-07-06T20:14:12.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.393+0000 7f62935ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0c6a60 con 0x7f628c0cac80 2023-07-06T20:14:12.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f6290b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 0x7f628c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f6290b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 0x7f628c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55120/0 (socket says 172.21.15.27:55120) 2023-07-06T20:14:12.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f6290b9a700 1 -- 172.21.15.27:0/2039618918 learned_addr learned my addr 172.21.15.27:0/2039618918 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.394+0000 7f629139b700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f6290b9a700 1 -- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f628c0cac80 msgr2=0x7f628c14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f6290b9a700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f628c0cac80 0x7f628c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f6290b9a700 1 -- 172.21.15.27:0/2039618918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 msgr2=0x7f628c0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f6290b9a700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f6290b9a700 1 -- 172.21.15.27:0/2039618918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280009550 con 0x7f628c070db0 2023-07-06T20:14:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae259c700 1 -- 172.21.15.27:0/600464419 >> v2:172.21.15.153:3300/0 conn(0x7f7ae40b2180 msgr2=0x7f7ae40cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae259c700 1 --2- 172.21.15.27:0/600464419 >> v2:172.21.15.153:3300/0 conn(0x7f7ae40b2180 0x7f7ae40cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae259c700 1 -- 172.21.15.27:0/600464419 >> v2:172.21.15.27:3300/0 conn(0x7f7ae40cfcb0 msgr2=0x7f7ae4160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae259c700 1 --2- 172.21.15.27:0/600464419 >> v2:172.21.15.27:3300/0 conn(0x7f7ae40cfcb0 0x7f7ae4160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f7ae259c700 1 -- 172.21.15.27:0/600464419 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40cd760 con 0x7f7ae40b8660 2023-07-06T20:14:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.395+0000 7f6290b9a700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 0x7f628c15aa20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f628400c8c0 tx=0x7f628400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a6c99c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a68160bc0 0x7f5a681540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a6c99c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a68160bc0 0x7f5a681540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44808/0 (socket says 172.21.15.27:44808) 2023-07-06T20:14:12.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a6c99c700 1 -- 172.21.15.27:0/2289914918 learned_addr learned my addr 172.21.15.27:0/2289914918 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a5f7fe700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a680c1870 0x7f5a680b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f7ae259c700 1 --2- 172.21.15.27:0/600464419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 0x7f7ae40ceed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406d000 tx=0x7f7adc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c6abba0d519be7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f404a9ff700 1 -- 172.21.15.27:0/2624938444 shutdown_connections 2023-07-06T20:14:12.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f4ab1599700 1 -- 172.21.15.27:0/3412342634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4010070 con 0x7f4aac0b0b50 2023-07-06T20:14:12.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f4ab1599700 1 -- 172.21.15.27:0/3412342634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa400dc90 con 0x7f4aac0b0b50 2023-07-06T20:14:12.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f4ab1599700 1 -- 172.21.15.27:0/3412342634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007b30 con 0x7f4aac0b0b50 2023-07-06T20:14:12.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a67fff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a680ad8f0 0x7f5a68151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a5f7fe700 1 -- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a680ad8f0 msgr2=0x7f5a68151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a5f7fe700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a680ad8f0 0x7f5a68151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a5f7fe700 1 -- 172.21.15.27:0/2289914918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a68160bc0 msgr2=0x7f5a681540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a5f7fe700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a68160bc0 0x7f5a681540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a5f7fe700 1 -- 172.21.15.27:0/2289914918 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500094a0 con 0x7f5a680c1870 2023-07-06T20:14:12.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f404a9ff700 1 -- 172.21.15.27:0/2624938444 wait complete. 2023-07-06T20:14:12.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.396+0000 7f5a67fff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a680ad8f0 0x7f5a68151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:12.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284003e10 con 0x7f628c070db0 2023-07-06T20:14:12.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400ce60 con 0x7f628c070db0 2023-07-06T20:14:12.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840052b0 con 0x7f628c070db0 2023-07-06T20:14:12.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0c6d40 con 0x7f628c070db0 2023-07-06T20:14:12.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f4ab47ff700 1 -- 172.21.15.27:0/3412342634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac0b0b50 msgr2=0x7f4aac163690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/3412342634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac0b0b50 0x7f4aac163690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400d370 tx=0x7f4aa400d680 comp rx=0 tx=0).stop 2023-07-06T20:14:12.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.398+0000 7f4ab47ff700 1 -- 172.21.15.27:0/3412342634 shutdown_connections 2023-07-06T20:14:12.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.398+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/3412342634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac0b0b50 0x7f4aac163690 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.398+0000 7f4ab47ff700 1 -- 172.21.15.27:0/3412342634 >> 172.21.15.27:0/3412342634 conn(0x7f4aac0cb440 msgr2=0x7f4aac0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f5a5f7fe700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a680c1870 0x7f5a680b37d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a50016000 tx=0x7f5a50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.397+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0c7a80 con 0x7f628c070db0 2023-07-06T20:14:12.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.398+0000 7f4ab47ff700 1 -- 172.21.15.27:0/3412342634 shutdown_connections 2023-07-06T20:14:12.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.398+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c0052f0 con 0x7f628c070db0 2023-07-06T20:14:12.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f404a9ff700 1 Processor -- start 2023-07-06T20:14:12.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a500036a0 con 0x7f5a680c1870 2023-07-06T20:14:12.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6814ec40 con 0x7f5a680c1870 2023-07-06T20:14:12.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5001f070 con 0x7f5a680c1870 2023-07-06T20:14:12.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50003a20 con 0x7f5a680c1870 2023-07-06T20:14:12.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6814e330 con 0x7f5a680c1870 2023-07-06T20:14:12.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.400+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a680008d0 con 0x7f5a680c1870 2023-07-06T20:14:12.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.400+0000 7f404a9ff700 1 -- start start 2023-07-06T20:14:12.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f492801a400 con 0x7f493806ec60 2023-07-06T20:14:12.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.399+0000 7f7ae159a700 1 -- 172.21.15.27:0/600464419 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00e070 con 0x7f7ae40b8660 2023-07-06T20:14:12.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.400+0000 7f7ae159a700 1 -- 172.21.15.27:0/600464419 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc00b910 con 0x7f7ae40b8660 2023-07-06T20:14:12.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.400+0000 7f7ae159a700 1 -- 172.21.15.27:0/600464419 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00bcc0 con 0x7f7ae40b8660 2023-07-06T20:14:12.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.402+0000 7f7ae8dff700 1 -- 172.21.15.27:0/600464419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 msgr2=0x7f7ae40ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.402+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/600464419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 0x7f7ae40ceed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406d000 tx=0x7f7adc00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:12.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f404a9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 0x7f40440b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f404a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f404414cf70 0x7f40440b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f404a9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40440b4250 0x7f404414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f404a9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440c0130 con 0x7f402c005320 2023-07-06T20:14:12.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f404a9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404414c760 con 0x7f40440b4250 2023-07-06T20:14:12.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f404a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40440b19b0 con 0x7f404414cf70 2023-07-06T20:14:12.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.401+0000 7f4ab47ff700 1 -- 172.21.15.27:0/3412342634 wait complete. 2023-07-06T20:14:12.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.402+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a50003b80 con 0x7f5a680c1870 2023-07-06T20:14:12.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.402+0000 7f491f7fe700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f492406adb0 0x7f492406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.403+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4928089a70 con 0x7f493806ec60 2023-07-06T20:14:12.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.403+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49280b53b0 con 0x7f493806ec60 2023-07-06T20:14:12.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.402+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6284005410 con 0x7f628c070db0 2023-07-06T20:14:12.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.403+0000 7f4043fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 0x7f40440b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.403+0000 7f4043fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 0x7f40440b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44810/0 (socket says 172.21.15.27:44810) 2023-07-06T20:14:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.403+0000 7f4043fff700 1 -- 172.21.15.27:0/2453016191 learned_addr learned my addr 172.21.15.27:0/2453016191 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f4048f9c700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40440b4250 0x7f404414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f40437fe700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f404414cf70 0x7f40440b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f4ab47ff700 1 Processor -- start 2023-07-06T20:14:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f49367fc700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f492406adb0 0x7f492406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f5a657fa700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f627dffb700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f627006ade0 0x7f627006d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6284088e00 con 0x7f628c070db0 2023-07-06T20:14:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a50021070 con 0x7f5a680c1870 2023-07-06T20:14:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f7ae8dff700 1 -- 172.21.15.27:0/600464419 shutdown_connections 2023-07-06T20:14:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/600464419 >> v2:172.21.15.27:3300/0 conn(0x7f7ae40cfcb0 0x7f7ae4160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/600464419 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b8660 0x7f7ae40ceed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/600464419 >> v2:172.21.15.153:3300/0 conn(0x7f7ae40b2180 0x7f7ae40cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f7ae8dff700 1 -- 172.21.15.27:0/600464419 >> 172.21.15.27:0/600464419 conn(0x7f7ae4071690 msgr2=0x7f7ae4071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f4043fff700 1 -- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f404414cf70 msgr2=0x7f40440b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f4043fff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f404414cf70 0x7f40440b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f4043fff700 1 -- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40440b4250 msgr2=0x7f404414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f4043fff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40440b4250 0x7f404414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f4043fff700 1 -- 172.21.15.27:0/2453016191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440b1c90 con 0x7f402c005320 2023-07-06T20:14:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.404+0000 7f4ab47ff700 1 -- start start 2023-07-06T20:14:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f4043fff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 0x7f40440b37d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f403400b3e0 tx=0x7f403400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.405+0000 7f5a67fff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f7ae8dff700 1 -- 172.21.15.27:0/600464419 shutdown_connections 2023-07-06T20:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403400bea0 con 0x7f402c005320 2023-07-06T20:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4034011850 con 0x7f402c005320 2023-07-06T20:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034011c00 con 0x7f402c005320 2023-07-06T20:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440b1f70 con 0x7f402c005320 2023-07-06T20:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404414dda0 con 0x7f402c005320 2023-07-06T20:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f49367fc700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f492406adb0 0x7f492406d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f492c0090b0 tx=0x7f492c014000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab47ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aac0b0b50 0x7f4aac1e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab47ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 0x7f4aac14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab47ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac1589e0 0x7f4aac156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab47ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac15a690 con 0x7f4aac1589e0 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab47ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1e2ff0 con 0x7f4aac0b0b50 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab47ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1e23e0 con 0x7f4aac158610 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab1d9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 0x7f4aac14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab1d9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 0x7f4aac14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:47794/0 (socket says 172.21.15.27:47794) 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f4ab1d9a700 1 -- 172.21.15.27:0/770035605 learned_addr learned my addr 172.21.15.27:0/770035605 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f4ab1d9a700 1 -- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aac0b0b50 msgr2=0x7f4aac1e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f4ab1d9a700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aac0b0b50 0x7f4aac1e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f4ab1d9a700 1 -- 172.21.15.27:0/770035605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac1589e0 msgr2=0x7f4aac156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f4ab1d9a700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac1589e0 0x7f4aac156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f4ab1d9a700 1 -- 172.21.15.27:0/770035605 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa400d050 con 0x7f4aac158610 2023-07-06T20:14:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.408+0000 7f4ab1d9a700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 0x7f4aac14d560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00b550 tx=0x7f4a9c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.407+0000 7f5a67fff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a6814bf00 tx=0x7f5a580073e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.406+0000 7f7ae8dff700 1 -- 172.21.15.27:0/600464419 wait complete. 2023-07-06T20:14:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f7ae8dff700 1 Processor -- start 2023-07-06T20:14:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f7ae8dff700 1 -- start start 2023-07-06T20:14:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f7ae8dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b2180 0x7f7ae4202d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f7ae8dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 0x7f7ae40c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f7ae8dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ae40cfcb0 0x7f7ae40b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f7ae8dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4203270 con 0x7f7ae40b8660 2023-07-06T20:14:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f7ae8dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406d660 con 0x7f7ae40cfcb0 2023-07-06T20:14:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f7ae8dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406d940 con 0x7f7ae40b2180 2023-07-06T20:14:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40440b1af0 con 0x7f402c005320 2023-07-06T20:14:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c011830 con 0x7f4aac158610 2023-07-06T20:14:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c0119b0 con 0x7f4aac158610 2023-07-06T20:14:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c011d60 con 0x7f4aac158610 2023-07-06T20:14:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac1e26c0 con 0x7f4aac158610 2023-07-06T20:14:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.409+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac1e2c80 con 0x7f4aac158610 2023-07-06T20:14:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac0008d0 con 0x7f4aac158610 2023-07-06T20:14:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.410+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a50055c20 con 0x7f5a680c1870 2023-07-06T20:14:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.408+0000 7f629139b700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f627006ade0 0x7f627006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.410+0000 7f629139b700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f627006ade0 0x7f627006d290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f628c16ab90 tx=0x7f6288008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.412+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6284053f20 con 0x7f628c070db0 2023-07-06T20:14:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.412+0000 7f7ae1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 0x7f7ae40c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.412+0000 7f7ae1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 0x7f7ae40c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44820/0 (socket says 172.21.15.27:44820) 2023-07-06T20:14:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.412+0000 7f7ae1d9b700 1 -- 172.21.15.27:0/2735760567 learned_addr learned my addr 172.21.15.27:0/2735760567 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.412+0000 7f7ae2d9d700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ae40cfcb0 0x7f7ae40b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f7ae1d9b700 1 -- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b2180 msgr2=0x7f7ae4202d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f7ae1d9b700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b2180 0x7f7ae4202d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f7ae1d9b700 1 -- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ae40cfcb0 msgr2=0x7f7ae40b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f7ae1d9b700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ae40cfcb0 0x7f7ae40b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f7ae1d9b700 1 -- 172.21.15.27:0/2735760567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc009b30 con 0x7f7ae40b8660 2023-07-06T20:14:12.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.414+0000 7f7ae1d9b700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 0x7f7ae40c8b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e870 tx=0x7f7ad400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.411+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f403401a400 con 0x7f402c005320 2023-07-06T20:14:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f4040ff9700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f403006adb0 0x7f403006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.413+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40340892e0 con 0x7f402c005320 2023-07-06T20:14:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.415+0000 7f40437fe700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f403006adb0 0x7f403006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.415+0000 7f40437fe700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f403006adb0 0x7f403006d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f403c009ee0 tx=0x7f403c009450 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.420+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f40440b1af0 con 0x7f402c005320 2023-07-06T20:14:12.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.417+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a9c01a400 con 0x7f4aac158610 2023-07-06T20:14:12.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.419+0000 7f4aa2ffd700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4a9006adb0 0x7f4a9006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.419+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a9c089b20 con 0x7f4aac158610 2023-07-06T20:14:12.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.420+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a9c0b53b0 con 0x7f4aac158610 2023-07-06T20:14:12.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.419+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400ee70 con 0x7f7ae40b8660 2023-07-06T20:14:12.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.419+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad400c850 con 0x7f7ae40b8660 2023-07-06T20:14:12.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.419+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400cbc0 con 0x7f7ae40b8660 2023-07-06T20:14:12.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.420+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae406f300 con 0x7f7ae40b8660 2023-07-06T20:14:12.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.420+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae415c330 con 0x7f7ae40b8660 2023-07-06T20:14:12.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.423+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ad400cd20 con 0x7f7ae40b8660 2023-07-06T20:14:12.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.424+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae40008d0 con 0x7f7ae40b8660 2023-07-06T20:14:12.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.425+0000 7f4ab259b700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4a9006adb0 0x7f4a9006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.425+0000 7f7ad2ffd700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7ac406adb0 0x7f7ac406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:12.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.425+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ad40893a0 con 0x7f7ae40b8660 2023-07-06T20:14:12.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.429+0000 7f7ae259c700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7ac406adb0 0x7f7ac406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:12.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.434+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ad4057f50 con 0x7f7ae40b8660 2023-07-06T20:14:12.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.442+0000 7f4ab259b700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4a9006adb0 0x7f4a9006d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8001600 tx=0x7f4aa8007a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.460+0000 7f7ae259c700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7ac406adb0 0x7f7ac406d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00b370 tx=0x7f7adc010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.748+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f41900b4940 con 0x7f4190158610 2023-07-06T20:14:12.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.748+0000 7f41867fc700 1 -- 172.21.15.27:0/117987168 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f418c05a770 con 0x7f4190158610 2023-07-06T20:14:12.751 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2023-07-06T20:14:12.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.749+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7b98161620 con 0x7f7b98160bc0 2023-07-06T20:14:12.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.750+0000 7f7b7f7fe700 1 -- 172.21.15.27:0/1542792974 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b90025380 con 0x7f7b98160bc0 2023-07-06T20:14:12.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.752+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f417406ad80 msgr2=0x7f417406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.752+0000 7f41981ff700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f417406ad80 0x7f417406d230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4180009710 tx=0x7f4180006c60 comp rx=0 tx=0).stop 2023-07-06T20:14:12.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.752+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 msgr2=0x7f41900c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.752+0000 7f41981ff700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 0x7f41900c7580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f418c018040 tx=0x7f418c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.752+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 shutdown_connections 2023-07-06T20:14:12.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f417406ad80 0x7f417406d230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41901589e0 0x7f41900c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41900caeb0 0x7f4190157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 --2- 172.21.15.27:0/117987168 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4190158610 0x7f41900c7580 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 >> 172.21.15.27:0/117987168 conn(0x7f41900cb440 msgr2=0x7f419006bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 shutdown_connections 2023-07-06T20:14:12.757 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-07-06T20:14:12.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.753+0000 7f41981ff700 1 -- 172.21.15.27:0/117987168 wait complete. 2023-07-06T20:14:12.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7b8406adb0 msgr2=0x7f7b8406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7b8406adb0 0x7f7b8406d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b88006b20 tx=0x7f7b88006ab0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 msgr2=0x7f7b98161920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98161920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000b370 tx=0x7f7b90004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 shutdown_connections 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7b8406adb0 0x7f7b8406d260 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b88006b20 tx=0x7f7b88006ab0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7b98160bc0 0x7f7b98161920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980ad8f0 0x7f7b980b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 --2- 172.21.15.27:0/1542792974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b980c1870 0x7f7b98153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.754+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 >> 172.21.15.27:0/1542792974 conn(0x7f7b980cb440 msgr2=0x7f7b980b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.756+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 shutdown_connections 2023-07-06T20:14:12.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.756+0000 7f7b9d1ff700 1 -- 172.21.15.27:0/1542792974 wait complete. 2023-07-06T20:14:12.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-07-06T20:14:12.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-07-06T20:14:12.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.824+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f626c005870 con 0x7f628c070db0 2023-07-06T20:14:12.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.825+0000 7f627dffb700 1 -- 172.21.15.27:0/2039618918 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6284017090 con 0x7f628c070db0 2023-07-06T20:14:12.827 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:12.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f627006ade0 msgr2=0x7f627006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f627006ade0 0x7f627006d290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f628c16ab90 tx=0x7f6288008040 comp rx=0 tx=0).stop 2023-07-06T20:14:12.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 msgr2=0x7f628c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 0x7f628c15aa20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f628400c8c0 tx=0x7f628400cc80 comp rx=0 tx=0).stop 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 shutdown_connections 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f627006ade0 0x7f627006d290 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f628c0cac80 0x7f628c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c06d800 0x7f628c0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 --2- 172.21.15.27:0/2039618918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f628c070db0 0x7f628c15aa20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.827+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 >> 172.21.15.27:0/2039618918 conn(0x7f628c0cb440 msgr2=0x7f628c158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.828+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 shutdown_connections 2023-07-06T20:14:12.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.828+0000 7f62935ff700 1 -- 172.21.15.27:0/2039618918 wait complete. 2023-07-06T20:14:12.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.828+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f40440c0270 con 0x7f402c005320 2023-07-06T20:14:12.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.828+0000 7f4040ff9700 1 -- 172.21.15.27:0/2453016191 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40440c0270 con 0x7f402c005320 2023-07-06T20:14:12.832 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2023-07-06T20:14:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f403006adb0 msgr2=0x7f403006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f403006adb0 0x7f403006d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f403c009ee0 tx=0x7f403c009450 comp rx=0 tx=0).stop 2023-07-06T20:14:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 msgr2=0x7f40440b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 0x7f40440b37d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f403400b3e0 tx=0x7f403400b6f0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 shutdown_connections 2023-07-06T20:14:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f403006adb0 0x7f403006d260 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f403c009ee0 tx=0x7f403c009450 comp rx=0 tx=0).stop 2023-07-06T20:14:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40440b4250 0x7f404414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c005320 0x7f40440b37d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.831+0000 7f404a9ff700 1 --2- 172.21.15.27:0/2453016191 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f404414cf70 0x7f40440b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.832+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 >> 172.21.15.27:0/2453016191 conn(0x7f40440cb440 msgr2=0x7f40440cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.832+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 shutdown_connections 2023-07-06T20:14:12.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.832+0000 7f404a9ff700 1 -- 172.21.15.27:0/2453016191 wait complete. 2023-07-06T20:14:12.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-07-06T20:14:12.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-07-06T20:14:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.875+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4918005870 con 0x7f493806ec60 2023-07-06T20:14:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.876+0000 7f491f7fe700 1 -- 172.21.15.27:0/3477886289 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4928054b00 con 0x7f493806ec60 2023-07-06T20:14:12.878 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:12.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.879+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f492406adb0 msgr2=0x7f492406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.879+0000 7f493d7ff700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f492406adb0 0x7f492406d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f492c0090b0 tx=0x7f492c014000 comp rx=0 tx=0).stop 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.879+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 msgr2=0x7f4938155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.879+0000 7f493d7ff700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 0x7f4938155fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f492800b550 tx=0x7f492800b860 comp rx=0 tx=0).stop 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 shutdown_connections 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f492406adb0 0x7f492406d260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4938159620 0x7f49380ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f493806ec60 0x7f4938155fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 --2- 172.21.15.27:0/3477886289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49380d0220 0x7f493815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 >> 172.21.15.27:0/3477886289 conn(0x7f49380b22f0 msgr2=0x7f49380cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 shutdown_connections 2023-07-06T20:14:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f493d7ff700 1 -- 172.21.15.27:0/3477886289 wait complete. 2023-07-06T20:14:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.880+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5a68071eb0 con 0x7f5a680c1870 2023-07-06T20:14:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.881+0000 7f5a657fa700 1 -- 172.21.15.27:0/2289914918 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a5002b090 con 0x7f5a680c1870 2023-07-06T20:14:12.884 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2023-07-06T20:14:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5a4806ace0 msgr2=0x7f5a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a6814bf00 tx=0x7f5a580073e0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a680c1870 msgr2=0x7f5a680b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a680c1870 0x7f5a680b37d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a50016000 tx=0x7f5a50004750 comp rx=0 tx=0).stop 2023-07-06T20:14:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 shutdown_connections 2023-07-06T20:14:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a68160bc0 0x7f5a681540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a680ad8f0 0x7f5a68151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 --2- 172.21.15.27:0/2289914918 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5a680c1870 0x7f5a680b37d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 >> 172.21.15.27:0/2289914918 conn(0x7f5a680cb440 msgr2=0x7f5a6806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 shutdown_connections 2023-07-06T20:14:12.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.884+0000 7f5a6e3ff700 1 -- 172.21.15.27:0/2289914918 wait complete. 2023-07-06T20:14:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.892+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4aac06fb10 con 0x7f4aac158610 2023-07-06T20:14:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.892+0000 7f4aa2ffd700 1 -- 172.21.15.27:0/770035605 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a9c015090 con 0x7f4aac158610 2023-07-06T20:14:12.894 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2023-07-06T20:14:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4a9006adb0 msgr2=0x7f4a9006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4a9006adb0 0x7f4a9006d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8001600 tx=0x7f4aa8007a20 comp rx=0 tx=0).stop 2023-07-06T20:14:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 msgr2=0x7f4aac14d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 0x7f4aac14d560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00b550 tx=0x7f4a9c00b910 comp rx=0 tx=0).stop 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 shutdown_connections 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4a9006adb0 0x7f4a9006d260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aac1589e0 0x7f4aac156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4aac0b0b50 0x7f4aac1e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 --2- 172.21.15.27:0/770035605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4aac158610 0x7f4aac14d560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 >> 172.21.15.27:0/770035605 conn(0x7f4aac0cb440 msgr2=0x7f4aac070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 shutdown_connections 2023-07-06T20:14:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.896+0000 7f4ab47ff700 1 -- 172.21.15.27:0/770035605 wait complete. 2023-07-06T20:14:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.897+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7ae40b5120 con 0x7f7ae40b8660 2023-07-06T20:14:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.898+0000 7f7ad2ffd700 1 -- 172.21.15.27:0/2735760567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ad4057ae0 con 0x7f7ae40b8660 2023-07-06T20:14:12.900 INFO:teuthology.orchestra.run.smithi027.stdout:55834574858 2023-07-06T20:14:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7ac406adb0 msgr2=0x7f7ac406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7ac406adb0 0x7f7ac406d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00b370 tx=0x7f7adc010000 comp rx=0 tx=0).stop 2023-07-06T20:14:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 msgr2=0x7f7ae40c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 0x7f7ae40c8b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e870 tx=0x7f7ad400ec30 comp rx=0 tx=0).stop 2023-07-06T20:14:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 shutdown_connections 2023-07-06T20:14:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7ac406adb0 0x7f7ac406d260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ae40cfcb0 0x7f7ae40b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ae40b2180 0x7f7ae4202d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 --2- 172.21.15.27:0/2735760567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ae40b8660 0x7f7ae40c8b90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.900+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 >> 172.21.15.27:0/2735760567 conn(0x7f7ae4071690 msgr2=0x7f7ae40b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.901+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 shutdown_connections 2023-07-06T20:14:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:12.901+0000 7f7ae8dff700 1 -- 172.21.15.27:0/2735760567 wait complete. 2023-07-06T20:14:12.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-07-06T20:14:12.908 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:12.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-07-06T20:14:12.912 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:12.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-07-06T20:14:12.919 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:12.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-07-06T20:14:12.922 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:13.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T20:14:13.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T20:14:13.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T20:14:13.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T20:14:14.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.027+0000 7f1553bff700 1 Processor -- start 2023-07-06T20:14:14.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.027+0000 7fb7a05ff700 1 Processor -- start 2023-07-06T20:14:14.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.027+0000 7fb7a05ff700 1 -- start start 2023-07-06T20:14:14.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.027+0000 7f1553bff700 1 -- start start 2023-07-06T20:14:14.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb7a05ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7980c9010 0x7fb7980c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb7a05ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7981565f0 0x7fb798156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f1553bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 0x7f154c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f1553bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c155220 con 0x7f154c0d1080 2023-07-06T20:14:14.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f1553bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c15bce0 con 0x7f154c06f9e0 2023-07-06T20:14:14.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f1553bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c06fe00 con 0x7f154c154e50 2023-07-06T20:14:14.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f155119a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f154c0d1080 0x7f154c06f2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36086/0 (socket says 172.21.15.27:36086) 2023-07-06T20:14:14.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f155119a700 1 -- 172.21.15.27:0/3269171418 learned_addr learned my addr 172.21.15.27:0/3269171418 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:14.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7f155199b700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 0x7f154c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb7a05ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7980c9a60 con 0x7fb79806c330 2023-07-06T20:14:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb7a05ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb798159b70 con 0x7fb7981565f0 2023-07-06T20:14:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb7a05ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb798159e50 con 0x7fb7980c9010 2023-07-06T20:14:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb79eb9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7981565f0 0x7fb798156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb79eb9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7981565f0 0x7fb798156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44826/0 (socket says 172.21.15.27:44826) 2023-07-06T20:14:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb79e39b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb79806c330 0x7fb79806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48734/0 (socket says 172.21.15.27:48734) 2023-07-06T20:14:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.029+0000 7fb79eb9c700 1 -- 172.21.15.27:0/1697245459 learned_addr learned my addr 172.21.15.27:0/1697245459 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79db9a700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7980c9010 0x7fb7980c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79eb9c700 1 -- 172.21.15.27:0/1697245459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7980c9010 msgr2=0x7fb7980c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348990389 0 0) 0x7f154c15bce0 con 0x7f154c06f9e0 2023-07-06T20:14:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15380035b0 con 0x7f154c06f9e0 2023-07-06T20:14:14.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430251857 0 0) 0x7f154c155220 con 0x7f154c0d1080 2023-07-06T20:14:14.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f154c15bce0 con 0x7f154c0d1080 2023-07-06T20:14:14.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79eb9c700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7980c9010 0x7fb7980c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79eb9c700 1 -- 172.21.15.27:0/1697245459 >> v1:172.21.15.153:6789/0 conn(0x7fb79806c330 legacy=0x7fb79806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79eb9c700 1 -- 172.21.15.27:0/1697245459 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79815a130 con 0x7fb7981565f0 2023-07-06T20:14:14.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79d399700 1 -- 172.21.15.27:0/1697245459 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584865266 0 0) 0x7fb7980c9a60 con 0x7fb79806c330 2023-07-06T20:14:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79db9a700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7980c9010 0x7fb7980c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546225170 0 0) 0x7f15380035b0 con 0x7f154c06f9e0 2023-07-06T20:14:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 msgr2=0x7f154c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7fb79eb9c700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7981565f0 0x7fb798156a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7900085d0 tx=0x7fb7900088e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1b7e4e67950ba6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 0x7f154c0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 >> v1:172.21.15.27:6789/0 conn(0x7f154c0d1080 legacy=0x7f154c06f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.030+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0ceab0 con 0x7f154c06f9e0 2023-07-06T20:14:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f1544001920 con 0x7f154c06f9e0 2023-07-06T20:14:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 >> v1:172.21.15.153:6789/0 conn(0x7f154c06f9e0 legacy=0x7f154c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1538004c20 0x7f1538004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 0x7f15380037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f15380041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c001230 con 0x7f1538003d30 2023-07-06T20:14:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c06fe00 con 0x7f1538004c20 2023-07-06T20:14:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153800fed0 con 0x7f154c154e50 2023-07-06T20:14:14.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7fb79d399700 1 -- 172.21.15.27:0/1697245459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000d2d0 con 0x7fb7981565f0 2023-07-06T20:14:14.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7fb79d399700 1 -- 172.21.15.27:0/1697245459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79000d430 con 0x7fb7981565f0 2023-07-06T20:14:14.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7fb79d399700 1 -- 172.21.15.27:0/1697245459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790016850 con 0x7fb7981565f0 2023-07-06T20:14:14.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f155219c700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f15380041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f155199b700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1538004c20 0x7f1538004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.031+0000 7f155119a700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 0x7f15380037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155219c700 1 -- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 msgr2=0x7f15380037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155219c700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c154e50 0x7f15380037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155219c700 1 -- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1538004c20 msgr2=0x7f1538004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155219c700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1538004c20 0x7f1538004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 -- 172.21.15.27:0/1697245459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7981565f0 msgr2=0x7fb798156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7981565f0 0x7fb798156a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7900085d0 tx=0x7fb7900088e0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155219c700 1 -- 172.21.15.27:0/3269171418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0ceab0 con 0x7f1538003d30 2023-07-06T20:14:14.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155199b700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1538004c20 0x7f1538004ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 -- 172.21.15.27:0/1697245459 shutdown_connections 2023-07-06T20:14:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7981565f0 0x7fb798156a60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/1697245459 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7980c9010 0x7fb7980c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 -- 172.21.15.27:0/1697245459 >> 172.21.15.27:0/1697245459 conn(0x7fb7980cb440 msgr2=0x7fb7980cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 -- 172.21.15.27:0/1697245459 shutdown_connections 2023-07-06T20:14:14.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7f155219c700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f15380041a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f154400a800 tx=0x7f1538005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329664a914fc4b0a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.032+0000 7fb7a05ff700 1 -- 172.21.15.27:0/1697245459 wait complete. 2023-07-06T20:14:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.033+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544012640 con 0x7f1538003d30 2023-07-06T20:14:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.033+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15440127a0 con 0x7f1538003d30 2023-07-06T20:14:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.033+0000 7f1550999700 1 -- 172.21.15.27:0/3269171418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544012b30 con 0x7f1538003d30 2023-07-06T20:14:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f1553bff700 1 -- 172.21.15.27:0/3269171418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 msgr2=0x7f15380041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f1553bff700 1 --2- 172.21.15.27:0/3269171418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f15380041a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f154400a800 tx=0x7f1538005880 comp rx=0 tx=0).stop 2023-07-06T20:14:14.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f155219c700 1 -- 172.21.15.27:0/3269171418 reap_dead start 2023-07-06T20:14:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f1553bff700 1 -- 172.21.15.27:0/3269171418 shutdown_connections 2023-07-06T20:14:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f1553bff700 1 -- 172.21.15.27:0/3269171418 >> 172.21.15.27:0/3269171418 conn(0x7f154c06db20 msgr2=0x7f154c06df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f1553bff700 1 -- 172.21.15.27:0/3269171418 shutdown_connections 2023-07-06T20:14:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7f1553bff700 1 -- 172.21.15.27:0/3269171418 wait complete. 2023-07-06T20:14:14.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.034+0000 7fb7a05ff700 1 Processor -- start 2023-07-06T20:14:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.035+0000 7fb7a05ff700 1 -- start start 2023-07-06T20:14:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7f1553bff700 1 Processor -- start 2023-07-06T20:14:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7fb7a05ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 0x7fb798155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7fb7a05ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7980c9010 0x7fb7980c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7f1553bff700 1 -- start start 2023-07-06T20:14:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7fb7a05ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7981565f0 0x7fb79815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7fb7a05ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798156050 con 0x7fb7980c9010 2023-07-06T20:14:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7fb7a05ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c7b80 con 0x7fb79806c330 2023-07-06T20:14:14.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.036+0000 7fb7a05ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c7e60 con 0x7fb7981565f0 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79e39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 0x7fb798155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79db9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7980c9010 0x7fb7980c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79e39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 0x7fb798155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55188/0 (socket says 172.21.15.27:55188) 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79e39b700 1 -- 172.21.15.27:0/314273743 learned_addr learned my addr 172.21.15.27:0/314273743 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79eb9c700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7981565f0 0x7fb79815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79eb9c700 1 -- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 msgr2=0x7fb798155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79eb9c700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 0x7fb798155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79eb9c700 1 -- 172.21.15.27:0/314273743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7980c9010 msgr2=0x7fb7980c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.037+0000 7fb79eb9c700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7980c9010 0x7fb7980c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7fb79eb9c700 1 -- 172.21.15.27:0/314273743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790008200 con 0x7fb7981565f0 2023-07-06T20:14:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7fb79e39b700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 0x7fb798155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7fb79db9a700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7980c9010 0x7fb7980c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f1553bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f154c0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f1553bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 0x7f154c0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f1553bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c0c8690 0x7f154c0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f1553bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0ca990 con 0x7f1538003d30 2023-07-06T20:14:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f1553bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0c9330 con 0x7f154c0c9fe0 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f1553bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0c9610 con 0x7f154c0c8690 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f155119a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 0x7f154c0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f155199b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f154c0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f155119a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 0x7f154c0ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55198/0 (socket says 172.21.15.27:55198) 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f155119a700 1 -- 172.21.15.27:0/2717239500 learned_addr learned my addr 172.21.15.27:0/2717239500 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7f155219c700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c0c8690 0x7f154c0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.038+0000 7fb79eb9c700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7981565f0 0x7fb79815ad50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb79001d000 tx=0x7fb790004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f155119a700 1 -- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c0c8690 msgr2=0x7f154c0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f155119a700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c0c8690 0x7f154c0c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f155119a700 1 -- 172.21.15.27:0/2717239500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 msgr2=0x7f154c0c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f155119a700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f154c0c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f155119a700 1 -- 172.21.15.27:0/2717239500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c155220 con 0x7f154c0c9fe0 2023-07-06T20:14:14.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790027070 con 0x7fb7981565f0 2023-07-06T20:14:14.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79000b590 con 0x7fb7981565f0 2023-07-06T20:14:14.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790003ea0 con 0x7fb7981565f0 2023-07-06T20:14:14.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980c8140 con 0x7fb7981565f0 2023-07-06T20:14:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7980c8700 con 0x7fb7981565f0 2023-07-06T20:14:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f155119a700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 0x7f154c0ca450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f153c007340 tx=0x7f153c007650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.039+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00f6c0 con 0x7f154c0c9fe0 2023-07-06T20:14:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.040+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c00f820 con 0x7f154c0c9fe0 2023-07-06T20:14:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.040+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00fb90 con 0x7f154c0c9fe0 2023-07-06T20:14:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.040+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c155500 con 0x7f154c0c9fe0 2023-07-06T20:14:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.040+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c0c9040 con 0x7f154c0c9fe0 2023-07-06T20:14:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.040+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7980c8940 con 0x7fb7981565f0 2023-07-06T20:14:14.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.041+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c155a20 con 0x7f154c0c9fe0 2023-07-06T20:14:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.042+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb79000b720 con 0x7fb7981565f0 2023-07-06T20:14:14.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.043+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f153c00fcf0 con 0x7f154c0c9fe0 2023-07-06T20:14:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.043+0000 7fb78effd700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77c06adb0 0x7fb77c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.043+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb790093640 con 0x7fb7981565f0 2023-07-06T20:14:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.044+0000 7fb79e39b700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77c06adb0 0x7fb77c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.044+0000 7f15427fc700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f153006ace0 0x7f153006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.044+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f153c089d80 con 0x7f154c0c9fe0 2023-07-06T20:14:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.044+0000 7fb79e39b700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77c06adb0 0x7fb77c06d260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb794000f60 tx=0x7fb794007df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.045+0000 7f155199b700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f153006ace0 0x7f153006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.045+0000 7f155199b700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f153006ace0 0x7f153006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1548009d80 tx=0x7f1548009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.047+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f153c054e10 con 0x7f154c0c9fe0 2023-07-06T20:14:14.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.048+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb7900620f0 con 0x7fb7981565f0 2023-07-06T20:14:14.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.116+0000 7f3ccfbff700 1 Processor -- start 2023-07-06T20:14:14.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.116+0000 7f3ccfbff700 1 -- start start 2023-07-06T20:14:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.117+0000 7f3ccfbff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3cc815fca0 0x7f3cc80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.117+0000 7f3ccfbff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3cc80b5c90 0x7f3cc80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.117+0000 7f3ccfbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b6110 0x7f3cc80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.117+0000 7f3ccfbff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80cf780 con 0x7f3cc815fca0 2023-07-06T20:14:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.117+0000 7f3ccfbff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80cd490 con 0x7f3cc80b5c90 2023-07-06T20:14:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.117+0000 7f3ccfbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80cd770 con 0x7f3cc80b6110 2023-07-06T20:14:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3cc815fca0 0x7f3cc80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3cc815fca0 0x7f3cc80d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44868/0 (socket says 172.21.15.27:44868) 2023-07-06T20:14:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 -- 172.21.15.27:0/1798368359 learned_addr learned my addr 172.21.15.27:0/1798368359 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd19a700 1 --2- 172.21.15.27:0/1798368359 >> v2:172.21.15.153:3300/0 conn(0x7f3cc80b5c90 0x7f3cc80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3cce19c700 1 --2- 172.21.15.27:0/1798368359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b6110 0x7f3cc80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 -- 172.21.15.27:0/1798368359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b6110 msgr2=0x7f3cc80d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/1798368359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b6110 0x7f3cc80d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 -- 172.21.15.27:0/1798368359 >> v2:172.21.15.153:3300/0 conn(0x7f3cc80b5c90 msgr2=0x7f3cc80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/1798368359 >> v2:172.21.15.153:3300/0 conn(0x7f3cc80b5c90 0x7f3cc80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.118+0000 7f3ccd99b700 1 -- 172.21.15.27:0/1798368359 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80cda50 con 0x7f3cc815fca0 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.119+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/1798368359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc80d1420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3cc806b6f0 tx=0x7f3cc4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a75796a234b350 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.119+0000 7f3ccc999700 1 -- 172.21.15.27:0/1798368359 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4011830 con 0x7f3cc815fca0 2023-07-06T20:14:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.119+0000 7f3ccc999700 1 -- 172.21.15.27:0/1798368359 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4011990 con 0x7f3cc815fca0 2023-07-06T20:14:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.119+0000 7f3ccc999700 1 -- 172.21.15.27:0/1798368359 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4011d20 con 0x7f3cc815fca0 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 -- 172.21.15.27:0/1798368359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 msgr2=0x7f3cc80d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/1798368359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc80d1420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3cc806b6f0 tx=0x7f3cc4009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 -- 172.21.15.27:0/1798368359 shutdown_connections 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/1798368359 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b6110 0x7f3cc80d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/1798368359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc80d1420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/1798368359 >> v2:172.21.15.153:3300/0 conn(0x7f3cc80b5c90 0x7f3cc80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 -- 172.21.15.27:0/1798368359 >> 172.21.15.27:0/1798368359 conn(0x7f3cc806c150 msgr2=0x7f3cc806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 -- 172.21.15.27:0/1798368359 shutdown_connections 2023-07-06T20:14:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.120+0000 7f3ccfbff700 1 -- 172.21.15.27:0/1798368359 wait complete. 2023-07-06T20:14:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.122+0000 7f3ccfbff700 1 Processor -- start 2023-07-06T20:14:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.122+0000 7f3ccfbff700 1 -- start start 2023-07-06T20:14:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f7de3fff700 1 Processor -- start 2023-07-06T20:14:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccfbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b5c90 0x7f3cc8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccfbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc80b6110 0x7f3cc81e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccfbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc81e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccfbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc81e6530 con 0x7f3cc815fca0 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccfbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b50a0 con 0x7f3cc80b6110 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccfbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b5380 con 0x7f3cc80b5c90 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3cce19c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc81e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3cce19c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc81e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44872/0 (socket says 172.21.15.27:44872) 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3cce19c700 1 -- 172.21.15.27:0/2133457050 learned_addr learned my addr 172.21.15.27:0/2133457050 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b5c90 0x7f3cc8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd19a700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc80b6110 0x7f3cc81e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd99b700 1 -- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc80b6110 msgr2=0x7f3cc81e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc80b6110 0x7f3cc81e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd99b700 1 -- 172.21.15.27:0/2133457050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 msgr2=0x7f3cc81e69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc81e69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd99b700 1 -- 172.21.15.27:0/2133457050 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009550 con 0x7f3cc80b5c90 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd19a700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc80b6110 0x7f3cc81e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3cce19c700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc81e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.124+0000 7f3ccd99b700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b5c90 0x7f3cc8156480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4016000 tx=0x7f3cc4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4004240 con 0x7f3cc80b5c90 2023-07-06T20:14:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc401f070 con 0x7f3cc80b5c90 2023-07-06T20:14:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc80b5660 con 0x7f3cc80b5c90 2023-07-06T20:14:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40045e0 con 0x7f3cc80b5c90 2023-07-06T20:14:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc81e61c0 con 0x7f3cc80b5c90 2023-07-06T20:14:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.123+0000 7f7de3fff700 1 -- start start 2023-07-06T20:14:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f7de3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ddc1591b0 0x7f7ddc0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f7de3fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ddc0cf5d0 0x7f7ddc06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f7de3fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cdff0 con 0x7f7ddc0cf5d0 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f7de3fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0ce2d0 con 0x7f7ddc1591b0 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.125+0000 7f7de3fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0ce5b0 con 0x7f7ddc159580 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ddc1591b0 0x7f7ddc0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ddc1591b0 0x7f7ddc0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55224/0 (socket says 172.21.15.27:55224) 2023-07-06T20:14:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 -- 172.21.15.27:0/3099252960 learned_addr learned my addr 172.21.15.27:0/3099252960 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de159a700 1 --2- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de259c700 1 --2- 172.21.15.27:0/3099252960 >> v2:172.21.15.27:3300/0 conn(0x7f7ddc0cf5d0 0x7f7ddc06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 -- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 msgr2=0x7f7ddc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 --2- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 -- 172.21.15.27:0/3099252960 >> v2:172.21.15.27:3300/0 conn(0x7f7ddc0cf5d0 msgr2=0x7f7ddc06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 --2- 172.21.15.27:0/3099252960 >> v2:172.21.15.27:3300/0 conn(0x7f7ddc0cf5d0 0x7f7ddc06b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.126+0000 7f7de1d9b700 1 -- 172.21.15.27:0/3099252960 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0ce890 con 0x7f7ddc1591b0 2023-07-06T20:14:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.127+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc80008d0 con 0x7f3cc80b5c90 2023-07-06T20:14:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.127+0000 7f7de1d9b700 1 --2- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc0cf090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc071a50 tx=0x7f7dd8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e251f64b5ce2ff5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.128+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3cc4021030 con 0x7f3cc80b5c90 2023-07-06T20:14:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.127+0000 7f7de0d99700 1 -- 172.21.15.27:0/3099252960 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8011830 con 0x7f7ddc1591b0 2023-07-06T20:14:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.127+0000 7f7de0d99700 1 -- 172.21.15.27:0/3099252960 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd8011990 con 0x7f7ddc1591b0 2023-07-06T20:14:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.127+0000 7f7de0d99700 1 -- 172.21.15.27:0/3099252960 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8011d20 con 0x7f7ddc1591b0 2023-07-06T20:14:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.128+0000 7f7de3fff700 1 -- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc1591b0 msgr2=0x7f7ddc0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.128+0000 7f7de3fff700 1 --2- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc0cf090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc071a50 tx=0x7f7dd8009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f3cbe7fc700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3cac06ace0 0x7f3cac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3cc408a5a0 con 0x7f3cc80b5c90 2023-07-06T20:14:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f3ccd19a700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3cac06ace0 0x7f3cac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 -- 172.21.15.27:0/3099252960 shutdown_connections 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 --2- 172.21.15.27:0/3099252960 >> v2:172.21.15.27:3300/0 conn(0x7f7ddc0cf5d0 0x7f7ddc06b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 --2- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc0cf090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 --2- 172.21.15.27:0/3099252960 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 -- 172.21.15.27:0/3099252960 >> 172.21.15.27:0/3099252960 conn(0x7f7ddc0cfaf0 msgr2=0x7f7ddc0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 -- 172.21.15.27:0/3099252960 shutdown_connections 2023-07-06T20:14:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.129+0000 7f7de3fff700 1 -- 172.21.15.27:0/3099252960 wait complete. 2023-07-06T20:14:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.130+0000 7f3ccd19a700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3cac06ace0 0x7f3cac06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009eb0 tx=0x7f3cb8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.132+0000 7f7de3fff700 1 Processor -- start 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.132+0000 7f7de3fff700 1 -- start start 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.133+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc0cf5d0 0x7f7ddc1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.133+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc14d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.133+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.133+0000 7f7de3fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc14db10 con 0x7f7ddc1591b0 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.133+0000 7f7de3fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc14e950 con 0x7f7ddc0cf5d0 2023-07-06T20:14:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.133+0000 7f7de3fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc14dd40 con 0x7f7ddc159580 2023-07-06T20:14:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3cc4055640 con 0x7f3cc80b5c90 2023-07-06T20:14:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc0cf5d0 0x7f7ddc1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc0cf5d0 0x7f7ddc1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55238/0 (socket says 172.21.15.27:55238) 2023-07-06T20:14:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de1d9b700 1 -- 172.21.15.27:0/151678047 learned_addr learned my addr 172.21.15.27:0/151678047 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc14d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de259c700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 -- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 msgr2=0x7f7ddc14ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc14ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 -- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc0cf5d0 msgr2=0x7f7ddc1526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc0cf5d0 0x7f7ddc1526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 -- 172.21.15.27:0/151678047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009550 con 0x7f7ddc1591b0 2023-07-06T20:14:14.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.134+0000 7f7de159a700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc14d5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00b240 tx=0x7f7dcc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.135+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00dc50 con 0x7f7ddc1591b0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.135+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc14e020 con 0x7f7ddc1591b0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.135+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc14e5e0 con 0x7f7ddc1591b0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.135+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc00eea0 con 0x7f7ddc1591b0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.135+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc005a20 con 0x7f7ddc1591b0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.136+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc40052f0 con 0x7f7ddc1591b0 2023-07-06T20:14:14.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.138+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7dcc005b80 con 0x7f7ddc1591b0 2023-07-06T20:14:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.139+0000 7f7dd27fc700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7dc006ad10 0x7f7dc006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.139+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7dcc089970 con 0x7f7ddc1591b0 2023-07-06T20:14:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.140+0000 7f7de1d9b700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7dc006ad10 0x7f7dc006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.140+0000 7f7de1d9b700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7dc006ad10 0x7f7dc006d1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009fd0 tx=0x7f7dd8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.148+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7dcc0583a0 con 0x7f7ddc1591b0 2023-07-06T20:14:14.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.349+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3cc80d06b0 con 0x7f3cc80b5c90 2023-07-06T20:14:14.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.350+0000 7f3cbe7fc700 1 -- 172.21.15.27:0/2133457050 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3cc4058c60 con 0x7f3cc80b5c90 2023-07-06T20:14:14.352 INFO:teuthology.orchestra.run.smithi027.stdout:55834574858 2023-07-06T20:14:14.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.353+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3cac06ace0 msgr2=0x7f3cac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.353+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3cac06ace0 0x7f3cac06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009eb0 tx=0x7f3cb8009210 comp rx=0 tx=0).stop 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.353+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b5c90 msgr2=0x7f3cc8156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.353+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b5c90 0x7f3cc8156480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4016000 tx=0x7f3cc4004750 comp rx=0 tx=0).stop 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 shutdown_connections 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3cac06ace0 0x7f3cac06d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cc815fca0 0x7f3cc81e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cc80b5c90 0x7f3cc8156480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 --2- 172.21.15.27:0/2133457050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc80b6110 0x7f3cc81e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 >> 172.21.15.27:0/2133457050 conn(0x7f3cc806c150 msgr2=0x7f3cc80d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 shutdown_connections 2023-07-06T20:14:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.354+0000 7f3ccfbff700 1 -- 172.21.15.27:0/2133457050 wait complete. 2023-07-06T20:14:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.367+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f154c0008d0 con 0x7f154c0c9fe0 2023-07-06T20:14:14.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.367+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb7980008d0 con 0x7fb7981565f0 2023-07-06T20:14:14.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.367+0000 7f15427fc700 1 -- 172.21.15.27:0/2717239500 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f153c058430 con 0x7f154c0c9fe0 2023-07-06T20:14:14.369 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:14.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.368+0000 7fb78effd700 1 -- 172.21.15.27:0/314273743 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb790061c80 con 0x7fb7981565f0 2023-07-06T20:14:14.370 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2023-07-06T20:14:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f153006ace0 msgr2=0x7f153006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f153006ace0 0x7f153006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1548009d80 tx=0x7f1548009210 comp rx=0 tx=0).stop 2023-07-06T20:14:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 msgr2=0x7f154c0ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 0x7f154c0ca450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f153c007340 tx=0x7f153c007650 comp rx=0 tx=0).stop 2023-07-06T20:14:14.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 shutdown_connections 2023-07-06T20:14:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f153006ace0 0x7f153006d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f154c0c8690 0x7f154c0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77c06adb0 msgr2=0x7fb77c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77c06adb0 0x7fb77c06d260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb794000f60 tx=0x7fb794007df0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1538003d30 0x7f154c0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 --2- 172.21.15.27:0/2717239500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f154c0c9fe0 0x7f154c0ca450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 >> 172.21.15.27:0/2717239500 conn(0x7f154c06db20 msgr2=0x7f154c06c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7981565f0 msgr2=0x7fb79815ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7981565f0 0x7fb79815ad50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb79001d000 tx=0x7fb790004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7dc40055a0 con 0x7f7ddc1591b0 2023-07-06T20:14:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 shutdown_connections 2023-07-06T20:14:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb77c06adb0 0x7fb77c06d260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb7981565f0 0x7fb79815ad50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.371+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 shutdown_connections 2023-07-06T20:14:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb79806c330 0x7fb798155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.370+0000 7fb7a05ff700 1 --2- 172.21.15.27:0/314273743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7980c9010 0x7fb7980c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.371+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 >> 172.21.15.27:0/314273743 conn(0x7fb7980cb440 msgr2=0x7fb79806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.371+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 shutdown_connections 2023-07-06T20:14:14.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.371+0000 7f1553bff700 1 -- 172.21.15.27:0/2717239500 wait complete. 2023-07-06T20:14:14.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.371+0000 7fb7a05ff700 1 -- 172.21.15.27:0/314273743 wait complete. 2023-07-06T20:14:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.371+0000 7f7dd27fc700 1 -- 172.21.15.27:0/151678047 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7dcc057f30 con 0x7f7ddc1591b0 2023-07-06T20:14:14.378 INFO:teuthology.orchestra.run.smithi027.stdout:55834574857 2023-07-06T20:14:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7dc006ad10 msgr2=0x7f7dc006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7dc006ad10 0x7f7dc006d1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009fd0 tx=0x7f7dd8008030 comp rx=0 tx=0).stop 2023-07-06T20:14:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ddc1591b0 msgr2=0x7f7ddc14d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc14d5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00b240 tx=0x7f7dcc00ec60 comp rx=0 tx=0).stop 2023-07-06T20:14:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 shutdown_connections 2023-07-06T20:14:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7dc006ad10 0x7f7dc006d1c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ddc159580 0x7f7ddc14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc0cf5d0 0x7f7ddc1526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.374+0000 7f7de3fff700 1 --2- 172.21.15.27:0/151678047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ddc1591b0 0x7f7ddc14d5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.375+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 >> 172.21.15.27:0/151678047 conn(0x7f7ddc0cfaf0 msgr2=0x7f7ddc06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.375+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 shutdown_connections 2023-07-06T20:14:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.375+0000 7f7de3fff700 1 -- 172.21.15.27:0/151678047 wait complete. 2023-07-06T20:14:14.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-07-06T20:14:14.382 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:14.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-07-06T20:14:14.396 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:14.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-07-06T20:14:14.397 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:14.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-07-06T20:14:14.400 DEBUG:teuthology.parallel:result is None 2023-07-06T20:14:14.400 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T20:14:14.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.643+0000 7f0ac2bff700 1 Processor -- start 2023-07-06T20:14:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.643+0000 7f0ac2bff700 1 -- start start 2023-07-06T20:14:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac2bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0abc15fca0 0x7f0abc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac2bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac2bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc0cf780 con 0x7f0abc0b60e0 2023-07-06T20:14:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac2bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc0cd490 con 0x7f0abc15fca0 2023-07-06T20:14:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac2bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0cd770 con 0x7f0abc0b5c90 2023-07-06T20:14:14.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac099b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0abc15fca0 0x7f0abc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac099b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0abc15fca0 0x7f0abc0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55246/0 (socket says 172.21.15.27:55246) 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0ac099b700 1 -- 172.21.15.27:0/933467697 learned_addr learned my addr 172.21.15.27:0/933467697 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.644+0000 7f0abbfff700 1 --2- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0ac099b700 1 -- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 msgr2=0x7f0abc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0ac099b700 1 --2- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0ac099b700 1 -- 172.21.15.27:0/933467697 >> v1:172.21.15.153:6789/0 conn(0x7f0abc0b60e0 legacy=0x7f0abc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0ac099b700 1 -- 172.21.15.27:0/933467697 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0cda50 con 0x7f0abc15fca0 2023-07-06T20:14:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0ac099b700 1 --2- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc15fca0 0x7f0abc0d1420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009e50 tx=0x7f0ab000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f684b96d2d576b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0abb7fe700 1 -- 172.21.15.27:0/933467697 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000e070 con 0x7f0abc15fca0 2023-07-06T20:14:14.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0abb7fe700 1 -- 172.21.15.27:0/933467697 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000b910 con 0x7f0abc15fca0 2023-07-06T20:14:14.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.645+0000 7f0abb7fe700 1 -- 172.21.15.27:0/933467697 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000bcc0 con 0x7f0abc15fca0 2023-07-06T20:14:14.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 -- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc15fca0 msgr2=0x7f0abc0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc15fca0 0x7f0abc0d1420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009e50 tx=0x7f0ab000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:14.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 -- 172.21.15.27:0/933467697 shutdown_connections 2023-07-06T20:14:14.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc15fca0 0x7f0abc0d1420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/933467697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 -- 172.21.15.27:0/933467697 >> 172.21.15.27:0/933467697 conn(0x7f0abc06c150 msgr2=0x7f0abc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 -- 172.21.15.27:0/933467697 shutdown_connections 2023-07-06T20:14:14.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.646+0000 7f0ac2bff700 1 -- 172.21.15.27:0/933467697 wait complete. 2023-07-06T20:14:14.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.648+0000 7f0ac2bff700 1 Processor -- start 2023-07-06T20:14:14.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.649+0000 7f0ac2bff700 1 -- start start 2023-07-06T20:14:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac2bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac2bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc0b60e0 0x7f0abc14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac2bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 0x7f0abc0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac2bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0cbf00 con 0x7f0abc15fca0 2023-07-06T20:14:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac2bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0cc3c0 con 0x7f0abc0b60e0 2023-07-06T20:14:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac2bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc0a99d0 con 0x7f0abc0b5c90 2023-07-06T20:14:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac119c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 0x7f0abc0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac119c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 0x7f0abc0b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44898/0 (socket says 172.21.15.27:44898) 2023-07-06T20:14:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac119c700 1 -- 172.21.15.27:0/846725813 learned_addr learned my addr 172.21.15.27:0/846725813 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.650+0000 7f0ac099b700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0abbfff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc0b60e0 0x7f0abc14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac119c700 1 -- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 msgr2=0x7f0abc0cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac119c700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac119c700 1 -- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc0b60e0 msgr2=0x7f0abc14d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac119c700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc0b60e0 0x7f0abc14d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac119c700 1 -- 172.21.15.27:0/846725813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0009b30 con 0x7f0abc15fca0 2023-07-06T20:14:14.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac119c700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 0x7f0abc0b7570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c530 tx=0x7f0ab400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40052d0 con 0x7f0abc15fca0 2023-07-06T20:14:14.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4005430 con 0x7f0abc15fca0 2023-07-06T20:14:14.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc15bf00 con 0x7f0abc15fca0 2023-07-06T20:14:14.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40038a0 con 0x7f0abc15fca0 2023-07-06T20:14:14.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.651+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc15c4c0 con 0x7f0abc15fca0 2023-07-06T20:14:14.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.652+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc0a9ba0 con 0x7f0abc15fca0 2023-07-06T20:14:14.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.653+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ab4003a00 con 0x7f0abc15fca0 2023-07-06T20:14:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.655+0000 7f0ab97fa700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0aa006ace0 0x7f0aa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.655+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ab40887a0 con 0x7f0abc15fca0 2023-07-06T20:14:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.655+0000 7f0ac099b700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0aa006ace0 0x7f0aa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.656+0000 7f0ac099b700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0aa006ace0 0x7f0aa006d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b370 tx=0x7f0ab0009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:14.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.659+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0abc0a9ba0 con 0x7f0abc15fca0 2023-07-06T20:14:14.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.863+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0abc06ae40 con 0x7f0aa006ace0 2023-07-06T20:14:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.871+0000 7f0ab97fa700 1 -- 172.21.15.27:0/846725813 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286615 (secure 0 0 0) 0x7f0ab0003b50 con 0x7f0aa006ace0 2023-07-06T20:14:14.876 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.875+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0aa006ace0 msgr2=0x7f0aa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.875+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0aa006ace0 0x7f0aa006d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b370 tx=0x7f0ab0009000 comp rx=0 tx=0).stop 2023-07-06T20:14:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.875+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 msgr2=0x7f0abc0b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.875+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 0x7f0abc0b7570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c530 tx=0x7f0ab400c840 comp rx=0 tx=0).stop 2023-07-06T20:14:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 shutdown_connections 2023-07-06T20:14:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0aa006ace0 0x7f0aa006d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0abc15fca0 0x7f0abc0b7570 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0abc0b5c90 0x7f0abc0cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 --2- 172.21.15.27:0/846725813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0abc0b60e0 0x7f0abc14d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 >> 172.21.15.27:0/846725813 conn(0x7f0abc06c150 msgr2=0x7f0abc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 shutdown_connections 2023-07-06T20:14:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:14.876+0000 7f0ac2bff700 1 -- 172.21.15.27:0/846725813 wait complete. 2023-07-06T20:14:14.880 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:14.897 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-07-06T20:14:14.346453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003079"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238965+0000","last_change":"2023-07-06T20:14:00.041890+0000","last_active":"2023-07-06T20:14:01.238965+0000","last_peered":"2023-07-06T20:14:01.238965+0000","last_clean":"2023-07-06T20:14:01.238965+0000","last_became_active":"2023-07-06T20:14:00.041518+0000","last_became_peered":"2023-07-06T20:14:00.041518+0000","last_unstale":"2023-07-06T20:14:01.238965+0000","last_undegraded":"2023-07-06T20:14:01.238965+0000","last_fullsized":"2023-07-06T20:14:01.238965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:26.527907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239844+0000","last_change":"2023-07-06T20:14:00.067412+0000","last_active":"2023-07-06T20:14:01.239844+0000","last_peered":"2023-07-06T20:14:01.239844+0000","last_clean":"2023-07-06T20:14:01.239844+0000","last_became_active":"2023-07-06T20:14:00.067104+0000","last_became_peered":"2023-07-06T20:14:00.067104+0000","last_unstale":"2023-07-06T20:14:01.239844+0000","last_undegraded":"2023-07-06T20:14:01.239844+0000","last_fullsized":"2023-07-06T20:14:01.239844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047383+0000","last_change":"2023-07-06T20:14:00.056588+0000","last_active":"2023-07-06T20:14:01.047383+0000","last_peered":"2023-07-06T20:14:01.047383+0000","last_clean":"2023-07-06T20:14:01.047383+0000","last_became_active":"2023-07-06T20:14:00.056267+0000","last_became_peered":"2023-07-06T20:14:00.056267+0000","last_unstale":"2023-07-06T20:14:01.047383+0000","last_undegraded":"2023-07-06T20:14:01.047383+0000","last_fullsized":"2023-07-06T20:14:01.047383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240208+0000","last_change":"2023-07-06T20:14:00.058980+0000","last_active":"2023-07-06T20:14:01.240208+0000","last_peered":"2023-07-06T20:14:01.240208+0000","last_clean":"2023-07-06T20:14:01.240208+0000","last_became_active":"2023-07-06T20:14:00.058707+0000","last_became_peered":"2023-07-06T20:14:00.058707+0000","last_unstale":"2023-07-06T20:14:01.240208+0000","last_undegraded":"2023-07-06T20:14:01.240208+0000","last_fullsized":"2023-07-06T20:14:01.240208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:14.285291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239684+0000","last_change":"2023-07-06T20:14:00.064519+0000","last_active":"2023-07-06T20:14:01.239684+0000","last_peered":"2023-07-06T20:14:01.239684+0000","last_clean":"2023-07-06T20:14:01.239684+0000","last_became_active":"2023-07-06T20:14:00.064199+0000","last_became_peered":"2023-07-06T20:14:00.064199+0000","last_unstale":"2023-07-06T20:14:01.239684+0000","last_undegraded":"2023-07-06T20:14:01.239684+0000","last_fullsized":"2023-07-06T20:14:01.239684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047036+0000","last_change":"2023-07-06T20:14:00.056564+0000","last_active":"2023-07-06T20:14:01.047036+0000","last_peered":"2023-07-06T20:14:01.047036+0000","last_clean":"2023-07-06T20:14:01.047036+0000","last_became_active":"2023-07-06T20:14:00.056292+0000","last_became_peered":"2023-07-06T20:14:00.056292+0000","last_unstale":"2023-07-06T20:14:01.047036+0000","last_undegraded":"2023-07-06T20:14:01.047036+0000","last_fullsized":"2023-07-06T20:14:01.047036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239166+0000","last_change":"2023-07-06T20:14:00.068005+0000","last_active":"2023-07-06T20:14:01.239166+0000","last_peered":"2023-07-06T20:14:01.239166+0000","last_clean":"2023-07-06T20:14:01.239166+0000","last_became_active":"2023-07-06T20:14:00.067567+0000","last_became_peered":"2023-07-06T20:14:00.067567+0000","last_unstale":"2023-07-06T20:14:01.239166+0000","last_undegraded":"2023-07-06T20:14:01.239166+0000","last_fullsized":"2023-07-06T20:14:01.239166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239597+0000","last_change":"2023-07-06T20:14:00.057310+0000","last_active":"2023-07-06T20:14:01.239597+0000","last_peered":"2023-07-06T20:14:01.239597+0000","last_clean":"2023-07-06T20:14:01.239597+0000","last_became_active":"2023-07-06T20:14:00.057048+0000","last_became_peered":"2023-07-06T20:14:00.057048+0000","last_unstale":"2023-07-06T20:14:01.239597+0000","last_undegraded":"2023-07-06T20:14:01.239597+0000","last_fullsized":"2023-07-06T20:14:01.239597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239261+0000","last_change":"2023-07-06T20:14:00.053189+0000","last_active":"2023-07-06T20:14:01.239261+0000","last_peered":"2023-07-06T20:14:01.239261+0000","last_clean":"2023-07-06T20:14:01.239261+0000","last_became_active":"2023-07-06T20:14:00.052649+0000","last_became_peered":"2023-07-06T20:14:00.052649+0000","last_unstale":"2023-07-06T20:14:01.239261+0000","last_undegraded":"2023-07-06T20:14:01.239261+0000","last_fullsized":"2023-07-06T20:14:01.239261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:44:22.477453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240134+0000","last_change":"2023-07-06T20:14:00.056320+0000","last_active":"2023-07-06T20:14:01.240134+0000","last_peered":"2023-07-06T20:14:01.240134+0000","last_clean":"2023-07-06T20:14:01.240134+0000","last_became_active":"2023-07-06T20:14:00.055864+0000","last_became_peered":"2023-07-06T20:14:00.055864+0000","last_unstale":"2023-07-06T20:14:01.240134+0000","last_undegraded":"2023-07-06T20:14:01.240134+0000","last_fullsized":"2023-07-06T20:14:01.240134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047499+0000","last_change":"2023-07-06T20:14:00.063394+0000","last_active":"2023-07-06T20:14:01.047499+0000","last_peered":"2023-07-06T20:14:01.047499+0000","last_clean":"2023-07-06T20:14:01.047499+0000","last_became_active":"2023-07-06T20:14:00.063126+0000","last_became_peered":"2023-07-06T20:14:00.063126+0000","last_unstale":"2023-07-06T20:14:01.047499+0000","last_undegraded":"2023-07-06T20:14:01.047499+0000","last_fullsized":"2023-07-06T20:14:01.047499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239136+0000","last_change":"2023-07-06T20:14:00.063773+0000","last_active":"2023-07-06T20:14:01.239136+0000","last_peered":"2023-07-06T20:14:01.239136+0000","last_clean":"2023-07-06T20:14:01.239136+0000","last_became_active":"2023-07-06T20:14:00.063498+0000","last_became_peered":"2023-07-06T20:14:00.063498+0000","last_unstale":"2023-07-06T20:14:01.239136+0000","last_undegraded":"2023-07-06T20:14:01.239136+0000","last_fullsized":"2023-07-06T20:14:01.239136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:24.723303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239028+0000","last_change":"2023-07-06T20:14:00.037831+0000","last_active":"2023-07-06T20:14:01.239028+0000","last_peered":"2023-07-06T20:14:01.239028+0000","last_clean":"2023-07-06T20:14:01.239028+0000","last_became_active":"2023-07-06T20:14:00.037512+0000","last_became_peered":"2023-07-06T20:14:00.037512+0000","last_unstale":"2023-07-06T20:14:01.239028+0000","last_undegraded":"2023-07-06T20:14:01.239028+0000","last_fullsized":"2023-07-06T20:14:01.239028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:17.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240002+0000","last_change":"2023-07-06T20:14:00.049995+0000","last_active":"2023-07-06T20:14:01.240002+0000","last_peered":"2023-07-06T20:14:01.240002+0000","last_clean":"2023-07-06T20:14:01.240002+0000","last_became_active":"2023-07-06T20:14:00.049548+0000","last_became_peered":"2023-07-06T20:14:00.049548+0000","last_unstale":"2023-07-06T20:14:01.240002+0000","last_undegraded":"2023-07-06T20:14:01.240002+0000","last_fullsized":"2023-07-06T20:14:01.240002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239729+0000","last_change":"2023-07-06T20:14:00.067827+0000","last_active":"2023-07-06T20:14:01.239729+0000","last_peered":"2023-07-06T20:14:01.239729+0000","last_clean":"2023-07-06T20:14:01.239729+0000","last_became_active":"2023-07-06T20:14:00.067219+0000","last_became_peered":"2023-07-06T20:14:00.067219+0000","last_unstale":"2023-07-06T20:14:01.239729+0000","last_undegraded":"2023-07-06T20:14:01.239729+0000","last_fullsized":"2023-07-06T20:14:01.239729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047086+0000","last_change":"2023-07-06T20:14:00.056312+0000","last_active":"2023-07-06T20:14:01.047086+0000","last_peered":"2023-07-06T20:14:01.047086+0000","last_clean":"2023-07-06T20:14:01.047086+0000","last_became_active":"2023-07-06T20:14:00.055971+0000","last_became_peered":"2023-07-06T20:14:00.055971+0000","last_unstale":"2023-07-06T20:14:01.047086+0000","last_undegraded":"2023-07-06T20:14:01.047086+0000","last_fullsized":"2023-07-06T20:14:01.047086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239406+0000","last_change":"2023-07-06T20:14:00.052988+0000","last_active":"2023-07-06T20:14:01.239406+0000","last_peered":"2023-07-06T20:14:01.239406+0000","last_clean":"2023-07-06T20:14:01.239406+0000","last_became_active":"2023-07-06T20:14:00.052324+0000","last_became_peered":"2023-07-06T20:14:00.052324+0000","last_unstale":"2023-07-06T20:14:01.239406+0000","last_undegraded":"2023-07-06T20:14:01.239406+0000","last_fullsized":"2023-07-06T20:14:01.239406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:32.142546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047643+0000","last_change":"2023-07-06T20:14:00.054064+0000","last_active":"2023-07-06T20:14:01.047643+0000","last_peered":"2023-07-06T20:14:01.047643+0000","last_clean":"2023-07-06T20:14:01.047643+0000","last_became_active":"2023-07-06T20:14:00.053459+0000","last_became_peered":"2023-07-06T20:14:00.053459+0000","last_unstale":"2023-07-06T20:14:01.047643+0000","last_undegraded":"2023-07-06T20:14:01.047643+0000","last_fullsized":"2023-07-06T20:14:01.047643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239803+0000","last_change":"2023-07-06T20:14:00.047280+0000","last_active":"2023-07-06T20:14:01.239803+0000","last_peered":"2023-07-06T20:14:01.239803+0000","last_clean":"2023-07-06T20:14:01.239803+0000","last_became_active":"2023-07-06T20:14:00.046943+0000","last_became_peered":"2023-07-06T20:14:00.046943+0000","last_unstale":"2023-07-06T20:14:01.239803+0000","last_undegraded":"2023-07-06T20:14:01.239803+0000","last_fullsized":"2023-07-06T20:14:01.239803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047192+0000","last_change":"2023-07-06T20:14:00.054317+0000","last_active":"2023-07-06T20:14:01.047192+0000","last_peered":"2023-07-06T20:14:01.047192+0000","last_clean":"2023-07-06T20:14:01.047192+0000","last_became_active":"2023-07-06T20:14:00.053672+0000","last_became_peered":"2023-07-06T20:14:00.053672+0000","last_unstale":"2023-07-06T20:14:01.047192+0000","last_undegraded":"2023-07-06T20:14:01.047192+0000","last_fullsized":"2023-07-06T20:14:01.047192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240189+0000","last_change":"2023-07-06T20:14:00.052959+0000","last_active":"2023-07-06T20:14:01.240189+0000","last_peered":"2023-07-06T20:14:01.240189+0000","last_clean":"2023-07-06T20:14:01.240189+0000","last_became_active":"2023-07-06T20:14:00.052605+0000","last_became_peered":"2023-07-06T20:14:00.052605+0000","last_unstale":"2023-07-06T20:14:01.240189+0000","last_undegraded":"2023-07-06T20:14:01.240189+0000","last_fullsized":"2023-07-06T20:14:01.240189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239515+0000","last_change":"2023-07-06T20:14:00.065147+0000","last_active":"2023-07-06T20:14:01.239515+0000","last_peered":"2023-07-06T20:14:01.239515+0000","last_clean":"2023-07-06T20:14:01.239515+0000","last_became_active":"2023-07-06T20:14:00.064652+0000","last_became_peered":"2023-07-06T20:14:00.064652+0000","last_unstale":"2023-07-06T20:14:01.239515+0000","last_undegraded":"2023-07-06T20:14:01.239515+0000","last_fullsized":"2023-07-06T20:14:01.239515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046578+0000","last_change":"2023-07-06T20:14:00.054578+0000","last_active":"2023-07-06T20:14:01.046578+0000","last_peered":"2023-07-06T20:14:01.046578+0000","last_clean":"2023-07-06T20:14:01.046578+0000","last_became_active":"2023-07-06T20:14:00.053689+0000","last_became_peered":"2023-07-06T20:14:00.053689+0000","last_unstale":"2023-07-06T20:14:01.046578+0000","last_undegraded":"2023-07-06T20:14:01.046578+0000","last_fullsized":"2023-07-06T20:14:01.046578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239287+0000","last_change":"2023-07-06T20:14:00.067375+0000","last_active":"2023-07-06T20:14:01.239287+0000","last_peered":"2023-07-06T20:14:01.239287+0000","last_clean":"2023-07-06T20:14:01.239287+0000","last_became_active":"2023-07-06T20:14:00.066909+0000","last_became_peered":"2023-07-06T20:14:00.066909+0000","last_unstale":"2023-07-06T20:14:01.239287+0000","last_undegraded":"2023-07-06T20:14:01.239287+0000","last_fullsized":"2023-07-06T20:14:01.239287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240385+0000","last_change":"2023-07-06T20:14:00.064820+0000","last_active":"2023-07-06T20:14:01.240385+0000","last_peered":"2023-07-06T20:14:01.240385+0000","last_clean":"2023-07-06T20:14:01.240385+0000","last_became_active":"2023-07-06T20:14:00.064585+0000","last_became_peered":"2023-07-06T20:14:00.064585+0000","last_unstale":"2023-07-06T20:14:01.240385+0000","last_undegraded":"2023-07-06T20:14:01.240385+0000","last_fullsized":"2023-07-06T20:14:01.240385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240518+0000","last_change":"2023-07-06T20:14:00.055061+0000","last_active":"2023-07-06T20:14:01.240518+0000","last_peered":"2023-07-06T20:14:01.240518+0000","last_clean":"2023-07-06T20:14:01.240518+0000","last_became_active":"2023-07-06T20:14:00.054795+0000","last_became_peered":"2023-07-06T20:14:00.054795+0000","last_unstale":"2023-07-06T20:14:01.240518+0000","last_undegraded":"2023-07-06T20:14:01.240518+0000","last_fullsized":"2023-07-06T20:14:01.240518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240667+0000","last_change":"2023-07-06T20:14:00.057631+0000","last_active":"2023-07-06T20:14:01.240667+0000","last_peered":"2023-07-06T20:14:01.240667+0000","last_clean":"2023-07-06T20:14:01.240667+0000","last_became_active":"2023-07-06T20:14:00.056582+0000","last_became_peered":"2023-07-06T20:14:00.056582+0000","last_unstale":"2023-07-06T20:14:01.240667+0000","last_undegraded":"2023-07-06T20:14:01.240667+0000","last_fullsized":"2023-07-06T20:14:01.240667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047284+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:14:01.047284+0000","last_peered":"2023-07-06T20:14:01.047284+0000","last_clean":"2023-07-06T20:14:01.047284+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:14:01.047284+0000","last_undegraded":"2023-07-06T20:14:01.047284+0000","last_fullsized":"2023-07-06T20:14:01.047284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047084+0000","last_change":"2023-07-06T20:14:00.067982+0000","last_active":"2023-07-06T20:14:01.047084+0000","last_peered":"2023-07-06T20:14:01.047084+0000","last_clean":"2023-07-06T20:14:01.047084+0000","last_became_active":"2023-07-06T20:14:00.067690+0000","last_became_peered":"2023-07-06T20:14:00.067690+0000","last_unstale":"2023-07-06T20:14:01.047084+0000","last_undegraded":"2023-07-06T20:14:01.047084+0000","last_fullsized":"2023-07-06T20:14:01.047084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239754+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:14:01.239754+0000","last_peered":"2023-07-06T20:14:01.239754+0000","last_clean":"2023-07-06T20:14:01.239754+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:14:01.239754+0000","last_undegraded":"2023-07-06T20:14:01.239754+0000","last_fullsized":"2023-07-06T20:14:01.239754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240204+0000","last_change":"2023-07-06T20:14:00.061405+0000","last_active":"2023-07-06T20:14:01.240204+0000","last_peered":"2023-07-06T20:14:01.240204+0000","last_clean":"2023-07-06T20:14:01.240204+0000","last_became_active":"2023-07-06T20:14:00.060981+0000","last_became_peered":"2023-07-06T20:14:00.060981+0000","last_unstale":"2023-07-06T20:14:01.240204+0000","last_undegraded":"2023-07-06T20:14:01.240204+0000","last_fullsized":"2023-07-06T20:14:01.240204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047136+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:14:01.047136+0000","last_peered":"2023-07-06T20:14:01.047136+0000","last_clean":"2023-07-06T20:14:01.047136+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:14:01.047136+0000","last_undegraded":"2023-07-06T20:14:01.047136+0000","last_fullsized":"2023-07-06T20:14:01.047136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239984+0000","last_change":"2023-07-06T20:14:00.052443+0000","last_active":"2023-07-06T20:14:01.239984+0000","last_peered":"2023-07-06T20:14:01.239984+0000","last_clean":"2023-07-06T20:14:01.239984+0000","last_became_active":"2023-07-06T20:14:00.052132+0000","last_became_peered":"2023-07-06T20:14:00.052132+0000","last_unstale":"2023-07-06T20:14:01.239984+0000","last_undegraded":"2023-07-06T20:14:01.239984+0000","last_fullsized":"2023-07-06T20:14:01.239984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:53.471708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239579+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:14:01.239579+0000","last_peered":"2023-07-06T20:14:01.239579+0000","last_clean":"2023-07-06T20:14:01.239579+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:14:01.239579+0000","last_undegraded":"2023-07-06T20:14:01.239579+0000","last_fullsized":"2023-07-06T20:14:01.239579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:14:00.067811+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:14:00.067528+0000","last_became_peered":"2023-07-06T20:14:00.067528+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239990+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:14:01.239990+0000","last_peered":"2023-07-06T20:14:01.239990+0000","last_clean":"2023-07-06T20:14:01.239990+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:14:01.239990+0000","last_undegraded":"2023-07-06T20:14:01.239990+0000","last_fullsized":"2023-07-06T20:14:01.239990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046539+0000","last_change":"2023-07-06T20:14:00.060824+0000","last_active":"2023-07-06T20:14:01.046539+0000","last_peered":"2023-07-06T20:14:01.046539+0000","last_clean":"2023-07-06T20:14:01.046539+0000","last_became_active":"2023-07-06T20:14:00.059809+0000","last_became_peered":"2023-07-06T20:14:00.059809+0000","last_unstale":"2023-07-06T20:14:01.046539+0000","last_undegraded":"2023-07-06T20:14:01.046539+0000","last_fullsized":"2023-07-06T20:14:01.046539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240120+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:14:01.240120+0000","last_peered":"2023-07-06T20:14:01.240120+0000","last_clean":"2023-07-06T20:14:01.240120+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.240120+0000","last_undegraded":"2023-07-06T20:14:01.240120+0000","last_fullsized":"2023-07-06T20:14:01.240120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:56.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240650+0000","last_change":"2023-07-06T20:14:00.056152+0000","last_active":"2023-07-06T20:14:01.240650+0000","last_peered":"2023-07-06T20:14:01.240650+0000","last_clean":"2023-07-06T20:14:01.240650+0000","last_became_active":"2023-07-06T20:14:00.055844+0000","last_became_peered":"2023-07-06T20:14:00.055844+0000","last_unstale":"2023-07-06T20:14:01.240650+0000","last_undegraded":"2023-07-06T20:14:01.240650+0000","last_fullsized":"2023-07-06T20:14:01.240650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047386+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:14:01.047386+0000","last_peered":"2023-07-06T20:14:01.047386+0000","last_clean":"2023-07-06T20:14:01.047386+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:14:01.047386+0000","last_undegraded":"2023-07-06T20:14:01.047386+0000","last_fullsized":"2023-07-06T20:14:01.047386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047231+0000","last_change":"2023-07-06T20:14:00.054939+0000","last_active":"2023-07-06T20:14:01.047231+0000","last_peered":"2023-07-06T20:14:01.047231+0000","last_clean":"2023-07-06T20:14:01.047231+0000","last_became_active":"2023-07-06T20:14:00.054354+0000","last_became_peered":"2023-07-06T20:14:00.054354+0000","last_unstale":"2023-07-06T20:14:01.047231+0000","last_undegraded":"2023-07-06T20:14:01.047231+0000","last_fullsized":"2023-07-06T20:14:01.047231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240033+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:14:01.240033+0000","last_peered":"2023-07-06T20:14:01.240033+0000","last_clean":"2023-07-06T20:14:01.240033+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:14:01.240033+0000","last_undegraded":"2023-07-06T20:14:01.240033+0000","last_fullsized":"2023-07-06T20:14:01.240033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:26.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046831+0000","last_change":"2023-07-06T20:14:00.056179+0000","last_active":"2023-07-06T20:14:01.046831+0000","last_peered":"2023-07-06T20:14:01.046831+0000","last_clean":"2023-07-06T20:14:01.046831+0000","last_became_active":"2023-07-06T20:14:00.055858+0000","last_became_peered":"2023-07-06T20:14:00.055858+0000","last_unstale":"2023-07-06T20:14:01.046831+0000","last_undegraded":"2023-07-06T20:14:01.046831+0000","last_fullsized":"2023-07-06T20:14:01.046831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239368+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:14:01.239368+0000","last_peered":"2023-07-06T20:14:01.239368+0000","last_clean":"2023-07-06T20:14:01.239368+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:14:01.239368+0000","last_undegraded":"2023-07-06T20:14:01.239368+0000","last_fullsized":"2023-07-06T20:14:01.239368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:48.108641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240807+0000","last_change":"2023-07-06T20:14:00.056298+0000","last_active":"2023-07-06T20:14:01.240807+0000","last_peered":"2023-07-06T20:14:01.240807+0000","last_clean":"2023-07-06T20:14:01.240807+0000","last_became_active":"2023-07-06T20:14:00.055932+0000","last_became_peered":"2023-07-06T20:14:00.055932+0000","last_unstale":"2023-07-06T20:14:01.240807+0000","last_undegraded":"2023-07-06T20:14:01.240807+0000","last_fullsized":"2023-07-06T20:14:01.240807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239561+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:14:01.239561+0000","last_peered":"2023-07-06T20:14:01.239561+0000","last_clean":"2023-07-06T20:14:01.239561+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:14:01.239561+0000","last_undegraded":"2023-07-06T20:14:01.239561+0000","last_fullsized":"2023-07-06T20:14:01.239561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240113+0000","last_change":"2023-07-06T20:14:00.064964+0000","last_active":"2023-07-06T20:14:01.240113+0000","last_peered":"2023-07-06T20:14:01.240113+0000","last_clean":"2023-07-06T20:14:01.240113+0000","last_became_active":"2023-07-06T20:14:00.064266+0000","last_became_peered":"2023-07-06T20:14:00.064266+0000","last_unstale":"2023-07-06T20:14:01.240113+0000","last_undegraded":"2023-07-06T20:14:01.240113+0000","last_fullsized":"2023-07-06T20:14:01.240113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239123+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:14:01.239123+0000","last_peered":"2023-07-06T20:14:01.239123+0000","last_clean":"2023-07-06T20:14:01.239123+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:14:01.239123+0000","last_undegraded":"2023-07-06T20:14:01.239123+0000","last_fullsized":"2023-07-06T20:14:01.239123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047315+0000","last_change":"2023-07-06T20:14:00.061173+0000","last_active":"2023-07-06T20:14:01.047315+0000","last_peered":"2023-07-06T20:14:01.047315+0000","last_clean":"2023-07-06T20:14:01.047315+0000","last_became_active":"2023-07-06T20:14:00.060346+0000","last_became_peered":"2023-07-06T20:14:00.060346+0000","last_unstale":"2023-07-06T20:14:01.047315+0000","last_undegraded":"2023-07-06T20:14:01.047315+0000","last_fullsized":"2023-07-06T20:14:01.047315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239999+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:14:01.239999+0000","last_peered":"2023-07-06T20:14:01.239999+0000","last_clean":"2023-07-06T20:14:01.239999+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:14:01.239999+0000","last_undegraded":"2023-07-06T20:14:01.239999+0000","last_fullsized":"2023-07-06T20:14:01.239999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:37.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239775+0000","last_change":"2023-07-06T20:14:00.061044+0000","last_active":"2023-07-06T20:14:01.239775+0000","last_peered":"2023-07-06T20:14:01.239775+0000","last_clean":"2023-07-06T20:14:01.239775+0000","last_became_active":"2023-07-06T20:14:00.060737+0000","last_became_peered":"2023-07-06T20:14:00.060737+0000","last_unstale":"2023-07-06T20:14:01.239775+0000","last_undegraded":"2023-07-06T20:14:01.239775+0000","last_fullsized":"2023-07-06T20:14:01.239775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240322+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:14:01.240322+0000","last_peered":"2023-07-06T20:14:01.240322+0000","last_clean":"2023-07-06T20:14:01.240322+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:14:01.240322+0000","last_undegraded":"2023-07-06T20:14:01.240322+0000","last_fullsized":"2023-07-06T20:14:01.240322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.062318+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.062063+0000","last_became_peered":"2023-07-06T20:14:00.062063+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240374+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:14:01.240374+0000","last_peered":"2023-07-06T20:14:01.240374+0000","last_clean":"2023-07-06T20:14:01.240374+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:14:01.240374+0000","last_undegraded":"2023-07-06T20:14:01.240374+0000","last_fullsized":"2023-07-06T20:14:01.240374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:11.129495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240536+0000","last_change":"2023-07-06T20:14:00.064988+0000","last_active":"2023-07-06T20:14:01.240536+0000","last_peered":"2023-07-06T20:14:01.240536+0000","last_clean":"2023-07-06T20:14:01.240536+0000","last_became_active":"2023-07-06T20:14:00.064581+0000","last_became_peered":"2023-07-06T20:14:00.064581+0000","last_unstale":"2023-07-06T20:14:01.240536+0000","last_undegraded":"2023-07-06T20:14:01.240536+0000","last_fullsized":"2023-07-06T20:14:01.240536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:10.399206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239427+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:14:01.239427+0000","last_peered":"2023-07-06T20:14:01.239427+0000","last_clean":"2023-07-06T20:14:01.239427+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:14:01.239427+0000","last_undegraded":"2023-07-06T20:14:01.239427+0000","last_fullsized":"2023-07-06T20:14:01.239427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:18.117668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239302+0000","last_change":"2023-07-06T20:14:00.065729+0000","last_active":"2023-07-06T20:14:01.239302+0000","last_peered":"2023-07-06T20:14:01.239302+0000","last_clean":"2023-07-06T20:14:01.239302+0000","last_became_active":"2023-07-06T20:14:00.064827+0000","last_became_peered":"2023-07-06T20:14:00.064827+0000","last_unstale":"2023-07-06T20:14:01.239302+0000","last_undegraded":"2023-07-06T20:14:01.239302+0000","last_fullsized":"2023-07-06T20:14:01.239302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:21:48.022430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046583+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:14:01.046583+0000","last_peered":"2023-07-06T20:14:01.046583+0000","last_clean":"2023-07-06T20:14:01.046583+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.046583+0000","last_undegraded":"2023-07-06T20:14:01.046583+0000","last_fullsized":"2023-07-06T20:14:01.046583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:18.188230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047260+0000","last_change":"2023-07-06T20:14:00.054542+0000","last_active":"2023-07-06T20:14:01.047260+0000","last_peered":"2023-07-06T20:14:01.047260+0000","last_clean":"2023-07-06T20:14:01.047260+0000","last_became_active":"2023-07-06T20:14:00.053987+0000","last_became_peered":"2023-07-06T20:14:00.053987+0000","last_unstale":"2023-07-06T20:14:01.047260+0000","last_undegraded":"2023-07-06T20:14:01.047260+0000","last_fullsized":"2023-07-06T20:14:01.047260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240427+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:14:01.240427+0000","last_peered":"2023-07-06T20:14:01.240427+0000","last_clean":"2023-07-06T20:14:01.240427+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:14:01.240427+0000","last_undegraded":"2023-07-06T20:14:01.240427+0000","last_fullsized":"2023-07-06T20:14:01.240427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:32.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047708+0000","last_change":"2023-07-06T20:14:00.052728+0000","last_active":"2023-07-06T20:14:01.047708+0000","last_peered":"2023-07-06T20:14:01.047708+0000","last_clean":"2023-07-06T20:14:01.047708+0000","last_became_active":"2023-07-06T20:14:00.052437+0000","last_became_peered":"2023-07-06T20:14:00.052437+0000","last_unstale":"2023-07-06T20:14:01.047708+0000","last_undegraded":"2023-07-06T20:14:01.047708+0000","last_fullsized":"2023-07-06T20:14:01.047708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:27.783430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047897+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:14:01.047897+0000","last_peered":"2023-07-06T20:14:01.047897+0000","last_clean":"2023-07-06T20:14:01.047897+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:14:01.047897+0000","last_undegraded":"2023-07-06T20:14:01.047897+0000","last_fullsized":"2023-07-06T20:14:01.047897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.053789+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.053148+0000","last_became_peered":"2023-07-06T20:14:00.053148+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:57.270521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046721+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:14:01.046721+0000","last_peered":"2023-07-06T20:14:01.046721+0000","last_clean":"2023-07-06T20:14:01.046721+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:14:01.046721+0000","last_undegraded":"2023-07-06T20:14:01.046721+0000","last_fullsized":"2023-07-06T20:14:01.046721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240548+0000","last_change":"2023-07-06T20:14:00.056310+0000","last_active":"2023-07-06T20:14:01.240548+0000","last_peered":"2023-07-06T20:14:01.240548+0000","last_clean":"2023-07-06T20:14:01.240548+0000","last_became_active":"2023-07-06T20:14:00.055916+0000","last_became_peered":"2023-07-06T20:14:00.055916+0000","last_unstale":"2023-07-06T20:14:01.240548+0000","last_undegraded":"2023-07-06T20:14:01.240548+0000","last_fullsized":"2023-07-06T20:14:01.240548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239879+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:14:01.239879+0000","last_peered":"2023-07-06T20:14:01.239879+0000","last_clean":"2023-07-06T20:14:01.239879+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:14:01.239879+0000","last_undegraded":"2023-07-06T20:14:01.239879+0000","last_fullsized":"2023-07-06T20:14:01.239879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:12.079947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240842+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:14:01.240842+0000","last_peered":"2023-07-06T20:14:01.240842+0000","last_clean":"2023-07-06T20:14:01.240842+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:14:01.240842+0000","last_undegraded":"2023-07-06T20:14:01.240842+0000","last_fullsized":"2023-07-06T20:14:01.240842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240741+0000","last_change":"2023-07-06T20:14:00.055724+0000","last_active":"2023-07-06T20:14:01.240741+0000","last_peered":"2023-07-06T20:14:01.240741+0000","last_clean":"2023-07-06T20:14:01.240741+0000","last_became_active":"2023-07-06T20:14:00.055440+0000","last_became_peered":"2023-07-06T20:14:00.055440+0000","last_unstale":"2023-07-06T20:14:01.240741+0000","last_undegraded":"2023-07-06T20:14:01.240741+0000","last_fullsized":"2023-07-06T20:14:01.240741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240130+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:14:01.240130+0000","last_peered":"2023-07-06T20:14:01.240130+0000","last_clean":"2023-07-06T20:14:01.240130+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:14:01.240130+0000","last_undegraded":"2023-07-06T20:14:01.240130+0000","last_fullsized":"2023-07-06T20:14:01.240130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:50.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047412+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:14:01.047412+0000","last_peered":"2023-07-06T20:14:01.047412+0000","last_clean":"2023-07-06T20:14:01.047412+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:14:01.047412+0000","last_undegraded":"2023-07-06T20:14:01.047412+0000","last_fullsized":"2023-07-06T20:14:01.047412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:17.962306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240019+0000","last_change":"2023-07-06T20:14:00.055529+0000","last_active":"2023-07-06T20:14:01.240019+0000","last_peered":"2023-07-06T20:14:01.240019+0000","last_clean":"2023-07-06T20:14:01.240019+0000","last_became_active":"2023-07-06T20:14:00.054757+0000","last_became_peered":"2023-07-06T20:14:00.054757+0000","last_unstale":"2023-07-06T20:14:01.240019+0000","last_undegraded":"2023-07-06T20:14:01.240019+0000","last_fullsized":"2023-07-06T20:14:01.240019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238739+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:14:01.238739+0000","last_peered":"2023-07-06T20:14:01.238739+0000","last_clean":"2023-07-06T20:14:01.238739+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:14:01.238739+0000","last_undegraded":"2023-07-06T20:14:01.238739+0000","last_fullsized":"2023-07-06T20:14:01.238739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239367+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:14:01.239367+0000","last_peered":"2023-07-06T20:14:01.239367+0000","last_clean":"2023-07-06T20:14:01.239367+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:14:01.239367+0000","last_undegraded":"2023-07-06T20:14:01.239367+0000","last_fullsized":"2023-07-06T20:14:01.239367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240403+0000","last_change":"2023-07-06T20:14:00.062155+0000","last_active":"2023-07-06T20:14:01.240403+0000","last_peered":"2023-07-06T20:14:01.240403+0000","last_clean":"2023-07-06T20:14:01.240403+0000","last_became_active":"2023-07-06T20:14:00.061848+0000","last_became_peered":"2023-07-06T20:14:00.061848+0000","last_unstale":"2023-07-06T20:14:01.240403+0000","last_undegraded":"2023-07-06T20:14:01.240403+0000","last_fullsized":"2023-07-06T20:14:01.240403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239665+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:14:01.239665+0000","last_peered":"2023-07-06T20:14:01.239665+0000","last_clean":"2023-07-06T20:14:01.239665+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:14:01.239665+0000","last_undegraded":"2023-07-06T20:14:01.239665+0000","last_fullsized":"2023-07-06T20:14:01.239665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:53.676445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239234+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:14:01.239234+0000","last_peered":"2023-07-06T20:14:01.239234+0000","last_clean":"2023-07-06T20:14:01.239234+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:14:01.239234+0000","last_undegraded":"2023-07-06T20:14:01.239234+0000","last_fullsized":"2023-07-06T20:14:01.239234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:04.461531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046254+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:14:01.046254+0000","last_peered":"2023-07-06T20:14:01.046254+0000","last_clean":"2023-07-06T20:14:01.046254+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:14:01.046254+0000","last_undegraded":"2023-07-06T20:14:01.046254+0000","last_fullsized":"2023-07-06T20:14:01.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239372+0000","last_change":"2023-07-06T20:14:00.062536+0000","last_active":"2023-07-06T20:14:01.239372+0000","last_peered":"2023-07-06T20:14:01.239372+0000","last_clean":"2023-07-06T20:14:01.239372+0000","last_became_active":"2023-07-06T20:14:00.062189+0000","last_became_peered":"2023-07-06T20:14:00.062189+0000","last_unstale":"2023-07-06T20:14:01.239372+0000","last_undegraded":"2023-07-06T20:14:01.239372+0000","last_fullsized":"2023-07-06T20:14:01.239372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:07.175614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239383+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:14:01.239383+0000","last_peered":"2023-07-06T20:14:01.239383+0000","last_clean":"2023-07-06T20:14:01.239383+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:14:01.239383+0000","last_undegraded":"2023-07-06T20:14:01.239383+0000","last_fullsized":"2023-07-06T20:14:01.239383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:38.150080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239126+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:14:01.239126+0000","last_peered":"2023-07-06T20:14:01.239126+0000","last_clean":"2023-07-06T20:14:01.239126+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:14:01.239126+0000","last_undegraded":"2023-07-06T20:14:01.239126+0000","last_fullsized":"2023-07-06T20:14:01.239126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:42.107407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240485+0000","last_change":"2023-07-06T20:14:00.065377+0000","last_active":"2023-07-06T20:14:01.240485+0000","last_peered":"2023-07-06T20:14:01.240485+0000","last_clean":"2023-07-06T20:14:01.240485+0000","last_became_active":"2023-07-06T20:14:00.064627+0000","last_became_peered":"2023-07-06T20:14:00.064627+0000","last_unstale":"2023-07-06T20:14:01.240485+0000","last_undegraded":"2023-07-06T20:14:01.240485+0000","last_fullsized":"2023-07-06T20:14:01.240485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240293+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:14:01.240293+0000","last_peered":"2023-07-06T20:14:01.240293+0000","last_clean":"2023-07-06T20:14:01.240293+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:14:01.240293+0000","last_undegraded":"2023-07-06T20:14:01.240293+0000","last_fullsized":"2023-07-06T20:14:01.240293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:41:59.625038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240045+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:14:01.240045+0000","last_peered":"2023-07-06T20:14:01.240045+0000","last_clean":"2023-07-06T20:14:01.240045+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:14:01.240045+0000","last_undegraded":"2023-07-06T20:14:01.240045+0000","last_fullsized":"2023-07-06T20:14:01.240045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239757+0000","last_change":"2023-07-06T20:14:00.057341+0000","last_active":"2023-07-06T20:14:01.239757+0000","last_peered":"2023-07-06T20:14:01.239757+0000","last_clean":"2023-07-06T20:14:01.239757+0000","last_became_active":"2023-07-06T20:14:00.057077+0000","last_became_peered":"2023-07-06T20:14:00.057077+0000","last_unstale":"2023-07-06T20:14:01.239757+0000","last_undegraded":"2023-07-06T20:14:01.239757+0000","last_fullsized":"2023-07-06T20:14:01.239757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047029+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:14:01.047029+0000","last_peered":"2023-07-06T20:14:01.047029+0000","last_clean":"2023-07-06T20:14:01.047029+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:14:01.047029+0000","last_undegraded":"2023-07-06T20:14:01.047029+0000","last_fullsized":"2023-07-06T20:14:01.047029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:37.193187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240442+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:14:01.240442+0000","last_peered":"2023-07-06T20:14:01.240442+0000","last_clean":"2023-07-06T20:14:01.240442+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:14:01.240442+0000","last_undegraded":"2023-07-06T20:14:01.240442+0000","last_fullsized":"2023-07-06T20:14:01.240442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:03.076475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046144+0000","last_change":"2023-07-06T20:14:00.066131+0000","last_active":"2023-07-06T20:14:01.046144+0000","last_peered":"2023-07-06T20:14:01.046144+0000","last_clean":"2023-07-06T20:14:01.046144+0000","last_became_active":"2023-07-06T20:14:00.065793+0000","last_became_peered":"2023-07-06T20:14:00.065793+0000","last_unstale":"2023-07-06T20:14:01.046144+0000","last_undegraded":"2023-07-06T20:14:01.046144+0000","last_fullsized":"2023-07-06T20:14:01.046144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:49.111646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239813+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:14:01.239813+0000","last_peered":"2023-07-06T20:14:01.239813+0000","last_clean":"2023-07-06T20:14:01.239813+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:14:01.239813+0000","last_undegraded":"2023-07-06T20:14:01.239813+0000","last_fullsized":"2023-07-06T20:14:01.239813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:17.841824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240384+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:14:01.240384+0000","last_peered":"2023-07-06T20:14:01.240384+0000","last_clean":"2023-07-06T20:14:01.240384+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:14:01.240384+0000","last_undegraded":"2023-07-06T20:14:01.240384+0000","last_fullsized":"2023-07-06T20:14:01.240384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047806+0000","last_change":"2023-07-06T20:14:00.055463+0000","last_active":"2023-07-06T20:14:01.047806+0000","last_peered":"2023-07-06T20:14:01.047806+0000","last_clean":"2023-07-06T20:14:01.047806+0000","last_became_active":"2023-07-06T20:14:00.055196+0000","last_became_peered":"2023-07-06T20:14:00.055196+0000","last_unstale":"2023-07-06T20:14:01.047806+0000","last_undegraded":"2023-07-06T20:14:01.047806+0000","last_fullsized":"2023-07-06T20:14:01.047806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:42:06.665764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239438+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:14:01.239438+0000","last_peered":"2023-07-06T20:14:01.239438+0000","last_clean":"2023-07-06T20:14:01.239438+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:14:01.239438+0000","last_undegraded":"2023-07-06T20:14:01.239438+0000","last_fullsized":"2023-07-06T20:14:01.239438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239401+0000","last_change":"2023-07-06T20:14:00.067006+0000","last_active":"2023-07-06T20:14:01.239401+0000","last_peered":"2023-07-06T20:14:01.239401+0000","last_clean":"2023-07-06T20:14:01.239401+0000","last_became_active":"2023-07-06T20:14:00.066735+0000","last_became_peered":"2023-07-06T20:14:00.066735+0000","last_unstale":"2023-07-06T20:14:01.239401+0000","last_undegraded":"2023-07-06T20:14:01.239401+0000","last_fullsized":"2023-07-06T20:14:01.239401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:37.878199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239281+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:14:01.239281+0000","last_peered":"2023-07-06T20:14:01.239281+0000","last_clean":"2023-07-06T20:14:01.239281+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:14:01.239281+0000","last_undegraded":"2023-07-06T20:14:01.239281+0000","last_fullsized":"2023-07-06T20:14:01.239281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047642+0000","last_change":"2023-07-06T20:14:00.054948+0000","last_active":"2023-07-06T20:14:01.047642+0000","last_peered":"2023-07-06T20:14:01.047642+0000","last_clean":"2023-07-06T20:14:01.047642+0000","last_became_active":"2023-07-06T20:14:00.054677+0000","last_became_peered":"2023-07-06T20:14:00.054677+0000","last_unstale":"2023-07-06T20:14:01.047642+0000","last_undegraded":"2023-07-06T20:14:01.047642+0000","last_fullsized":"2023-07-06T20:14:01.047642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:38.225314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240958+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:14:01.240958+0000","last_peered":"2023-07-06T20:14:01.240958+0000","last_clean":"2023-07-06T20:14:01.240958+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:14:01.240958+0000","last_undegraded":"2023-07-06T20:14:01.240958+0000","last_fullsized":"2023-07-06T20:14:01.240958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:37.736933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047478+0000","last_change":"2023-07-06T20:14:00.053372+0000","last_active":"2023-07-06T20:14:01.047478+0000","last_peered":"2023-07-06T20:14:01.047478+0000","last_clean":"2023-07-06T20:14:01.047478+0000","last_became_active":"2023-07-06T20:14:00.053043+0000","last_became_peered":"2023-07-06T20:14:00.053043+0000","last_unstale":"2023-07-06T20:14:01.047478+0000","last_undegraded":"2023-07-06T20:14:01.047478+0000","last_fullsized":"2023-07-06T20:14:01.047478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046413+0000","last_change":"2023-07-06T20:14:00.038980+0000","last_active":"2023-07-06T20:14:01.046413+0000","last_peered":"2023-07-06T20:14:01.046413+0000","last_clean":"2023-07-06T20:14:01.046413+0000","last_became_active":"2023-07-06T20:14:00.038706+0000","last_became_peered":"2023-07-06T20:14:00.038706+0000","last_unstale":"2023-07-06T20:14:01.046413+0000","last_undegraded":"2023-07-06T20:14:01.046413+0000","last_fullsized":"2023-07-06T20:14:01.046413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239873+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:14:01.239873+0000","last_peered":"2023-07-06T20:14:01.239873+0000","last_clean":"2023-07-06T20:14:01.239873+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:14:01.239873+0000","last_undegraded":"2023-07-06T20:14:01.239873+0000","last_fullsized":"2023-07-06T20:14:01.239873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047021+0000","last_change":"2023-07-06T20:14:00.049540+0000","last_active":"2023-07-06T20:14:01.047021+0000","last_peered":"2023-07-06T20:14:01.047021+0000","last_clean":"2023-07-06T20:14:01.047021+0000","last_became_active":"2023-07-06T20:14:00.049260+0000","last_became_peered":"2023-07-06T20:14:00.049260+0000","last_unstale":"2023-07-06T20:14:01.047021+0000","last_undegraded":"2023-07-06T20:14:01.047021+0000","last_fullsized":"2023-07-06T20:14:01.047021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239288+0000","last_change":"2023-07-06T20:14:00.048736+0000","last_active":"2023-07-06T20:14:01.239288+0000","last_peered":"2023-07-06T20:14:01.239288+0000","last_clean":"2023-07-06T20:14:01.239288+0000","last_became_active":"2023-07-06T20:14:00.048418+0000","last_became_peered":"2023-07-06T20:14:00.048418+0000","last_unstale":"2023-07-06T20:14:01.239288+0000","last_undegraded":"2023-07-06T20:14:01.239288+0000","last_fullsized":"2023-07-06T20:14:01.239288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046375+0000","last_change":"2023-07-06T20:14:00.067909+0000","last_active":"2023-07-06T20:14:01.046375+0000","last_peered":"2023-07-06T20:14:01.046375+0000","last_clean":"2023-07-06T20:14:01.046375+0000","last_became_active":"2023-07-06T20:14:00.067600+0000","last_became_peered":"2023-07-06T20:14:00.067600+0000","last_unstale":"2023-07-06T20:14:01.046375+0000","last_undegraded":"2023-07-06T20:14:01.046375+0000","last_fullsized":"2023-07-06T20:14:01.046375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047258+0000","last_change":"2023-07-06T20:14:00.058103+0000","last_active":"2023-07-06T20:14:01.047258+0000","last_peered":"2023-07-06T20:14:01.047258+0000","last_clean":"2023-07-06T20:14:01.047258+0000","last_became_active":"2023-07-06T20:14:00.057800+0000","last_became_peered":"2023-07-06T20:14:00.057800+0000","last_unstale":"2023-07-06T20:14:01.047258+0000","last_undegraded":"2023-07-06T20:14:01.047258+0000","last_fullsized":"2023-07-06T20:14:01.047258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046528+0000","last_change":"2023-07-06T20:14:00.061079+0000","last_active":"2023-07-06T20:14:01.046528+0000","last_peered":"2023-07-06T20:14:01.046528+0000","last_clean":"2023-07-06T20:14:01.046528+0000","last_became_active":"2023-07-06T20:14:00.059768+0000","last_became_peered":"2023-07-06T20:14:00.059768+0000","last_unstale":"2023-07-06T20:14:01.046528+0000","last_undegraded":"2023-07-06T20:14:01.046528+0000","last_fullsized":"2023-07-06T20:14:01.046528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239855+0000","last_change":"2023-07-06T20:14:00.056153+0000","last_active":"2023-07-06T20:14:01.239855+0000","last_peered":"2023-07-06T20:14:01.239855+0000","last_clean":"2023-07-06T20:14:01.239855+0000","last_became_active":"2023-07-06T20:14:00.055830+0000","last_became_peered":"2023-07-06T20:14:00.055830+0000","last_unstale":"2023-07-06T20:14:01.239855+0000","last_undegraded":"2023-07-06T20:14:01.239855+0000","last_fullsized":"2023-07-06T20:14:01.239855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.665}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.208}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.997999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.247999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.0010000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7939999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1710000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:14.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:14:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.139+0000 7f7433fff700 1 Processor -- start 2023-07-06T20:14:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.139+0000 7f7433fff700 1 -- start start 2023-07-06T20:14:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.140+0000 7f7433fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.140+0000 7f7433fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f742c0b6100 0x7f742c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.140+0000 7f7433fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c0cf780 con 0x7f742c0b5c90 2023-07-06T20:14:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.140+0000 7f7433fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c0cd490 con 0x7f742c0b6100 2023-07-06T20:14:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.140+0000 7f7433fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0cd770 con 0x7f742c15fca0 2023-07-06T20:14:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743159a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f742c0b5c90 0x7f742c0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36092/0 (socket says 172.21.15.27:36092) 2023-07-06T20:14:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743159a700 1 -- 172.21.15.27:0/638087676 learned_addr learned my addr 172.21.15.27:0/638087676 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743259c700 1 --2- 172.21.15.27:0/638087676 >> v2:172.21.15.27:3300/0 conn(0x7f742c0b6100 0x7f742c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f7431d9b700 1 --2- 172.21.15.27:0/638087676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f7430d99700 1 -- 172.21.15.27:0/638087676 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054829075 0 0) 0x7f742c0cf780 con 0x7f742c0b5c90 2023-07-06T20:14:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f7430d99700 1 -- 172.21.15.27:0/638087676 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74180035b0 con 0x7f742c0b5c90 2023-07-06T20:14:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743259c700 1 -- 172.21.15.27:0/638087676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 msgr2=0x7f742c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743259c700 1 --2- 172.21.15.27:0/638087676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743259c700 1 -- 172.21.15.27:0/638087676 >> v1:172.21.15.27:6789/0 conn(0x7f742c0b5c90 legacy=0x7f742c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743259c700 1 -- 172.21.15.27:0/638087676 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c0cda50 con 0x7f742c0b6100 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.141+0000 7f743259c700 1 --2- 172.21.15.27:0/638087676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b6100 0x7f742c0cfc10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7424009d90 tx=0x7f742400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dae87e9840971e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7430d99700 1 -- 172.21.15.27:0/638087676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400e070 con 0x7f742c0b6100 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7430d99700 1 -- 172.21.15.27:0/638087676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742400bb20 con 0x7f742c0b6100 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7430d99700 1 -- 172.21.15.27:0/638087676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424012430 con 0x7f742c0b6100 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 -- 172.21.15.27:0/638087676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b6100 msgr2=0x7f742c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 --2- 172.21.15.27:0/638087676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b6100 0x7f742c0cfc10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7424009d90 tx=0x7f742400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 -- 172.21.15.27:0/638087676 shutdown_connections 2023-07-06T20:14:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 --2- 172.21.15.27:0/638087676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b6100 0x7f742c0cfc10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 --2- 172.21.15.27:0/638087676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 -- 172.21.15.27:0/638087676 >> 172.21.15.27:0/638087676 conn(0x7f742c06c150 msgr2=0x7f742c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 -- 172.21.15.27:0/638087676 shutdown_connections 2023-07-06T20:14:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.142+0000 7f7433fff700 1 -- 172.21.15.27:0/638087676 wait complete. 2023-07-06T20:14:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.144+0000 7f7433fff700 1 Processor -- start 2023-07-06T20:14:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.144+0000 7f7433fff700 1 -- start start 2023-07-06T20:14:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7433fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 0x7f742c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7433fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f742c0b6100 0x7f742c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7433fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7433fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c15e9b0 con 0x7f742c0b5c90 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7433fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c15b730 con 0x7f742c0b6100 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7433fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0cc460 con 0x7f742c15fca0 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7431d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 0x7f742c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7431d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 0x7f742c0cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44914/0 (socket says 172.21.15.27:44914) 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.145+0000 7f7431d9b700 1 -- 172.21.15.27:0/2555056262 learned_addr learned my addr 172.21.15.27:0/2555056262 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f743259c700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f743159a700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f742c0b6100 0x7f742c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7431d9b700 1 -- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 msgr2=0x7f742c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7431d9b700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7431d9b700 1 -- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f742c0b6100 msgr2=0x7f742c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7431d9b700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f742c0b6100 0x7f742c157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7431d9b700 1 -- 172.21.15.27:0/2555056262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009a70 con 0x7f742c0b5c90 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7431d9b700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 0x7f742c0cbe40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f742800d800 tx=0x7f742800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280125f0 con 0x7f742c0b5c90 2023-07-06T20:14:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428012750 con 0x7f742c0b5c90 2023-07-06T20:14:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428012b00 con 0x7f742c0b5c90 2023-07-06T20:14:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.146+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c0cc740 con 0x7f742c0b5c90 2023-07-06T20:14:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.147+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c15b980 con 0x7f742c0b5c90 2023-07-06T20:14:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.148+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c15bb40 con 0x7f742c0b5c90 2023-07-06T20:14:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.148+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7428012c60 con 0x7f742c0b5c90 2023-07-06T20:14:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.149+0000 7f74227fc700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f741006ace0 0x7f741006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.149+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74280897d0 con 0x7f742c0b5c90 2023-07-06T20:14:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.150+0000 7f743159a700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f741006ace0 0x7f741006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.151+0000 7f743159a700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f741006ace0 0x7f741006d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f742c0d04d0 tx=0x7f741c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:15.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.153+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7428054910 con 0x7f742c0b5c90 2023-07-06T20:14:15.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.357+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f742c06ae40 con 0x7f741006ace0 2023-07-06T20:14:15.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.366+0000 7f74227fc700 1 -- 172.21.15.27:0/2555056262 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286615 (secure 0 0 0) 0x7f741c0058e0 con 0x7f741006ace0 2023-07-06T20:14:15.371 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:15.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f741006ace0 msgr2=0x7f741006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f741006ace0 0x7f741006d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f742c0d04d0 tx=0x7f741c009040 comp rx=0 tx=0).stop 2023-07-06T20:14:15.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 msgr2=0x7f742c0cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 0x7f742c0cbe40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f742800d800 tx=0x7f742800dbc0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 shutdown_connections 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f741006ace0 0x7f741006d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f742c15fca0 0x7f742c0d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f742c0b5c90 0x7f742c0cbe40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 --2- 172.21.15.27:0/2555056262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f742c0b6100 0x7f742c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 >> 172.21.15.27:0/2555056262 conn(0x7f742c06c150 msgr2=0x7f742c15a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:15.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 shutdown_connections 2023-07-06T20:14:15.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.370+0000 7f7433fff700 1 -- 172.21.15.27:0/2555056262 wait complete. 2023-07-06T20:14:15.374 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:14:15.391 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-07-06T20:14:14.346453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003079"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238965+0000","last_change":"2023-07-06T20:14:00.041890+0000","last_active":"2023-07-06T20:14:01.238965+0000","last_peered":"2023-07-06T20:14:01.238965+0000","last_clean":"2023-07-06T20:14:01.238965+0000","last_became_active":"2023-07-06T20:14:00.041518+0000","last_became_peered":"2023-07-06T20:14:00.041518+0000","last_unstale":"2023-07-06T20:14:01.238965+0000","last_undegraded":"2023-07-06T20:14:01.238965+0000","last_fullsized":"2023-07-06T20:14:01.238965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:26.527907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239844+0000","last_change":"2023-07-06T20:14:00.067412+0000","last_active":"2023-07-06T20:14:01.239844+0000","last_peered":"2023-07-06T20:14:01.239844+0000","last_clean":"2023-07-06T20:14:01.239844+0000","last_became_active":"2023-07-06T20:14:00.067104+0000","last_became_peered":"2023-07-06T20:14:00.067104+0000","last_unstale":"2023-07-06T20:14:01.239844+0000","last_undegraded":"2023-07-06T20:14:01.239844+0000","last_fullsized":"2023-07-06T20:14:01.239844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047383+0000","last_change":"2023-07-06T20:14:00.056588+0000","last_active":"2023-07-06T20:14:01.047383+0000","last_peered":"2023-07-06T20:14:01.047383+0000","last_clean":"2023-07-06T20:14:01.047383+0000","last_became_active":"2023-07-06T20:14:00.056267+0000","last_became_peered":"2023-07-06T20:14:00.056267+0000","last_unstale":"2023-07-06T20:14:01.047383+0000","last_undegraded":"2023-07-06T20:14:01.047383+0000","last_fullsized":"2023-07-06T20:14:01.047383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240208+0000","last_change":"2023-07-06T20:14:00.058980+0000","last_active":"2023-07-06T20:14:01.240208+0000","last_peered":"2023-07-06T20:14:01.240208+0000","last_clean":"2023-07-06T20:14:01.240208+0000","last_became_active":"2023-07-06T20:14:00.058707+0000","last_became_peered":"2023-07-06T20:14:00.058707+0000","last_unstale":"2023-07-06T20:14:01.240208+0000","last_undegraded":"2023-07-06T20:14:01.240208+0000","last_fullsized":"2023-07-06T20:14:01.240208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:14.285291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239684+0000","last_change":"2023-07-06T20:14:00.064519+0000","last_active":"2023-07-06T20:14:01.239684+0000","last_peered":"2023-07-06T20:14:01.239684+0000","last_clean":"2023-07-06T20:14:01.239684+0000","last_became_active":"2023-07-06T20:14:00.064199+0000","last_became_peered":"2023-07-06T20:14:00.064199+0000","last_unstale":"2023-07-06T20:14:01.239684+0000","last_undegraded":"2023-07-06T20:14:01.239684+0000","last_fullsized":"2023-07-06T20:14:01.239684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047036+0000","last_change":"2023-07-06T20:14:00.056564+0000","last_active":"2023-07-06T20:14:01.047036+0000","last_peered":"2023-07-06T20:14:01.047036+0000","last_clean":"2023-07-06T20:14:01.047036+0000","last_became_active":"2023-07-06T20:14:00.056292+0000","last_became_peered":"2023-07-06T20:14:00.056292+0000","last_unstale":"2023-07-06T20:14:01.047036+0000","last_undegraded":"2023-07-06T20:14:01.047036+0000","last_fullsized":"2023-07-06T20:14:01.047036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239166+0000","last_change":"2023-07-06T20:14:00.068005+0000","last_active":"2023-07-06T20:14:01.239166+0000","last_peered":"2023-07-06T20:14:01.239166+0000","last_clean":"2023-07-06T20:14:01.239166+0000","last_became_active":"2023-07-06T20:14:00.067567+0000","last_became_peered":"2023-07-06T20:14:00.067567+0000","last_unstale":"2023-07-06T20:14:01.239166+0000","last_undegraded":"2023-07-06T20:14:01.239166+0000","last_fullsized":"2023-07-06T20:14:01.239166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239597+0000","last_change":"2023-07-06T20:14:00.057310+0000","last_active":"2023-07-06T20:14:01.239597+0000","last_peered":"2023-07-06T20:14:01.239597+0000","last_clean":"2023-07-06T20:14:01.239597+0000","last_became_active":"2023-07-06T20:14:00.057048+0000","last_became_peered":"2023-07-06T20:14:00.057048+0000","last_unstale":"2023-07-06T20:14:01.239597+0000","last_undegraded":"2023-07-06T20:14:01.239597+0000","last_fullsized":"2023-07-06T20:14:01.239597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239261+0000","last_change":"2023-07-06T20:14:00.053189+0000","last_active":"2023-07-06T20:14:01.239261+0000","last_peered":"2023-07-06T20:14:01.239261+0000","last_clean":"2023-07-06T20:14:01.239261+0000","last_became_active":"2023-07-06T20:14:00.052649+0000","last_became_peered":"2023-07-06T20:14:00.052649+0000","last_unstale":"2023-07-06T20:14:01.239261+0000","last_undegraded":"2023-07-06T20:14:01.239261+0000","last_fullsized":"2023-07-06T20:14:01.239261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:44:22.477453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240134+0000","last_change":"2023-07-06T20:14:00.056320+0000","last_active":"2023-07-06T20:14:01.240134+0000","last_peered":"2023-07-06T20:14:01.240134+0000","last_clean":"2023-07-06T20:14:01.240134+0000","last_became_active":"2023-07-06T20:14:00.055864+0000","last_became_peered":"2023-07-06T20:14:00.055864+0000","last_unstale":"2023-07-06T20:14:01.240134+0000","last_undegraded":"2023-07-06T20:14:01.240134+0000","last_fullsized":"2023-07-06T20:14:01.240134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047499+0000","last_change":"2023-07-06T20:14:00.063394+0000","last_active":"2023-07-06T20:14:01.047499+0000","last_peered":"2023-07-06T20:14:01.047499+0000","last_clean":"2023-07-06T20:14:01.047499+0000","last_became_active":"2023-07-06T20:14:00.063126+0000","last_became_peered":"2023-07-06T20:14:00.063126+0000","last_unstale":"2023-07-06T20:14:01.047499+0000","last_undegraded":"2023-07-06T20:14:01.047499+0000","last_fullsized":"2023-07-06T20:14:01.047499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239136+0000","last_change":"2023-07-06T20:14:00.063773+0000","last_active":"2023-07-06T20:14:01.239136+0000","last_peered":"2023-07-06T20:14:01.239136+0000","last_clean":"2023-07-06T20:14:01.239136+0000","last_became_active":"2023-07-06T20:14:00.063498+0000","last_became_peered":"2023-07-06T20:14:00.063498+0000","last_unstale":"2023-07-06T20:14:01.239136+0000","last_undegraded":"2023-07-06T20:14:01.239136+0000","last_fullsized":"2023-07-06T20:14:01.239136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:36:24.723303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239028+0000","last_change":"2023-07-06T20:14:00.037831+0000","last_active":"2023-07-06T20:14:01.239028+0000","last_peered":"2023-07-06T20:14:01.239028+0000","last_clean":"2023-07-06T20:14:01.239028+0000","last_became_active":"2023-07-06T20:14:00.037512+0000","last_became_peered":"2023-07-06T20:14:00.037512+0000","last_unstale":"2023-07-06T20:14:01.239028+0000","last_undegraded":"2023-07-06T20:14:01.239028+0000","last_fullsized":"2023-07-06T20:14:01.239028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:49:17.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240002+0000","last_change":"2023-07-06T20:14:00.049995+0000","last_active":"2023-07-06T20:14:01.240002+0000","last_peered":"2023-07-06T20:14:01.240002+0000","last_clean":"2023-07-06T20:14:01.240002+0000","last_became_active":"2023-07-06T20:14:00.049548+0000","last_became_peered":"2023-07-06T20:14:00.049548+0000","last_unstale":"2023-07-06T20:14:01.240002+0000","last_undegraded":"2023-07-06T20:14:01.240002+0000","last_fullsized":"2023-07-06T20:14:01.240002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239729+0000","last_change":"2023-07-06T20:14:00.067827+0000","last_active":"2023-07-06T20:14:01.239729+0000","last_peered":"2023-07-06T20:14:01.239729+0000","last_clean":"2023-07-06T20:14:01.239729+0000","last_became_active":"2023-07-06T20:14:00.067219+0000","last_became_peered":"2023-07-06T20:14:00.067219+0000","last_unstale":"2023-07-06T20:14:01.239729+0000","last_undegraded":"2023-07-06T20:14:01.239729+0000","last_fullsized":"2023-07-06T20:14:01.239729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047086+0000","last_change":"2023-07-06T20:14:00.056312+0000","last_active":"2023-07-06T20:14:01.047086+0000","last_peered":"2023-07-06T20:14:01.047086+0000","last_clean":"2023-07-06T20:14:01.047086+0000","last_became_active":"2023-07-06T20:14:00.055971+0000","last_became_peered":"2023-07-06T20:14:00.055971+0000","last_unstale":"2023-07-06T20:14:01.047086+0000","last_undegraded":"2023-07-06T20:14:01.047086+0000","last_fullsized":"2023-07-06T20:14:01.047086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239406+0000","last_change":"2023-07-06T20:14:00.052988+0000","last_active":"2023-07-06T20:14:01.239406+0000","last_peered":"2023-07-06T20:14:01.239406+0000","last_clean":"2023-07-06T20:14:01.239406+0000","last_became_active":"2023-07-06T20:14:00.052324+0000","last_became_peered":"2023-07-06T20:14:00.052324+0000","last_unstale":"2023-07-06T20:14:01.239406+0000","last_undegraded":"2023-07-06T20:14:01.239406+0000","last_fullsized":"2023-07-06T20:14:01.239406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:32.142546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047643+0000","last_change":"2023-07-06T20:14:00.054064+0000","last_active":"2023-07-06T20:14:01.047643+0000","last_peered":"2023-07-06T20:14:01.047643+0000","last_clean":"2023-07-06T20:14:01.047643+0000","last_became_active":"2023-07-06T20:14:00.053459+0000","last_became_peered":"2023-07-06T20:14:00.053459+0000","last_unstale":"2023-07-06T20:14:01.047643+0000","last_undegraded":"2023-07-06T20:14:01.047643+0000","last_fullsized":"2023-07-06T20:14:01.047643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239803+0000","last_change":"2023-07-06T20:14:00.047280+0000","last_active":"2023-07-06T20:14:01.239803+0000","last_peered":"2023-07-06T20:14:01.239803+0000","last_clean":"2023-07-06T20:14:01.239803+0000","last_became_active":"2023-07-06T20:14:00.046943+0000","last_became_peered":"2023-07-06T20:14:00.046943+0000","last_unstale":"2023-07-06T20:14:01.239803+0000","last_undegraded":"2023-07-06T20:14:01.239803+0000","last_fullsized":"2023-07-06T20:14:01.239803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047192+0000","last_change":"2023-07-06T20:14:00.054317+0000","last_active":"2023-07-06T20:14:01.047192+0000","last_peered":"2023-07-06T20:14:01.047192+0000","last_clean":"2023-07-06T20:14:01.047192+0000","last_became_active":"2023-07-06T20:14:00.053672+0000","last_became_peered":"2023-07-06T20:14:00.053672+0000","last_unstale":"2023-07-06T20:14:01.047192+0000","last_undegraded":"2023-07-06T20:14:01.047192+0000","last_fullsized":"2023-07-06T20:14:01.047192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240189+0000","last_change":"2023-07-06T20:14:00.052959+0000","last_active":"2023-07-06T20:14:01.240189+0000","last_peered":"2023-07-06T20:14:01.240189+0000","last_clean":"2023-07-06T20:14:01.240189+0000","last_became_active":"2023-07-06T20:14:00.052605+0000","last_became_peered":"2023-07-06T20:14:00.052605+0000","last_unstale":"2023-07-06T20:14:01.240189+0000","last_undegraded":"2023-07-06T20:14:01.240189+0000","last_fullsized":"2023-07-06T20:14:01.240189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239515+0000","last_change":"2023-07-06T20:14:00.065147+0000","last_active":"2023-07-06T20:14:01.239515+0000","last_peered":"2023-07-06T20:14:01.239515+0000","last_clean":"2023-07-06T20:14:01.239515+0000","last_became_active":"2023-07-06T20:14:00.064652+0000","last_became_peered":"2023-07-06T20:14:00.064652+0000","last_unstale":"2023-07-06T20:14:01.239515+0000","last_undegraded":"2023-07-06T20:14:01.239515+0000","last_fullsized":"2023-07-06T20:14:01.239515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046578+0000","last_change":"2023-07-06T20:14:00.054578+0000","last_active":"2023-07-06T20:14:01.046578+0000","last_peered":"2023-07-06T20:14:01.046578+0000","last_clean":"2023-07-06T20:14:01.046578+0000","last_became_active":"2023-07-06T20:14:00.053689+0000","last_became_peered":"2023-07-06T20:14:00.053689+0000","last_unstale":"2023-07-06T20:14:01.046578+0000","last_undegraded":"2023-07-06T20:14:01.046578+0000","last_fullsized":"2023-07-06T20:14:01.046578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239287+0000","last_change":"2023-07-06T20:14:00.067375+0000","last_active":"2023-07-06T20:14:01.239287+0000","last_peered":"2023-07-06T20:14:01.239287+0000","last_clean":"2023-07-06T20:14:01.239287+0000","last_became_active":"2023-07-06T20:14:00.066909+0000","last_became_peered":"2023-07-06T20:14:00.066909+0000","last_unstale":"2023-07-06T20:14:01.239287+0000","last_undegraded":"2023-07-06T20:14:01.239287+0000","last_fullsized":"2023-07-06T20:14:01.239287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240385+0000","last_change":"2023-07-06T20:14:00.064820+0000","last_active":"2023-07-06T20:14:01.240385+0000","last_peered":"2023-07-06T20:14:01.240385+0000","last_clean":"2023-07-06T20:14:01.240385+0000","last_became_active":"2023-07-06T20:14:00.064585+0000","last_became_peered":"2023-07-06T20:14:00.064585+0000","last_unstale":"2023-07-06T20:14:01.240385+0000","last_undegraded":"2023-07-06T20:14:01.240385+0000","last_fullsized":"2023-07-06T20:14:01.240385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240518+0000","last_change":"2023-07-06T20:14:00.055061+0000","last_active":"2023-07-06T20:14:01.240518+0000","last_peered":"2023-07-06T20:14:01.240518+0000","last_clean":"2023-07-06T20:14:01.240518+0000","last_became_active":"2023-07-06T20:14:00.054795+0000","last_became_peered":"2023-07-06T20:14:00.054795+0000","last_unstale":"2023-07-06T20:14:01.240518+0000","last_undegraded":"2023-07-06T20:14:01.240518+0000","last_fullsized":"2023-07-06T20:14:01.240518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:57.165938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240667+0000","last_change":"2023-07-06T20:14:00.057631+0000","last_active":"2023-07-06T20:14:01.240667+0000","last_peered":"2023-07-06T20:14:01.240667+0000","last_clean":"2023-07-06T20:14:01.240667+0000","last_became_active":"2023-07-06T20:14:00.056582+0000","last_became_peered":"2023-07-06T20:14:00.056582+0000","last_unstale":"2023-07-06T20:14:01.240667+0000","last_undegraded":"2023-07-06T20:14:01.240667+0000","last_fullsized":"2023-07-06T20:14:01.240667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047284+0000","last_change":"2023-07-06T20:13:59.024119+0000","last_active":"2023-07-06T20:14:01.047284+0000","last_peered":"2023-07-06T20:14:01.047284+0000","last_clean":"2023-07-06T20:14:01.047284+0000","last_became_active":"2023-07-06T20:13:59.023815+0000","last_became_peered":"2023-07-06T20:13:59.023815+0000","last_unstale":"2023-07-06T20:14:01.047284+0000","last_undegraded":"2023-07-06T20:14:01.047284+0000","last_fullsized":"2023-07-06T20:14:01.047284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:09.114196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047084+0000","last_change":"2023-07-06T20:14:00.067982+0000","last_active":"2023-07-06T20:14:01.047084+0000","last_peered":"2023-07-06T20:14:01.047084+0000","last_clean":"2023-07-06T20:14:01.047084+0000","last_became_active":"2023-07-06T20:14:00.067690+0000","last_became_peered":"2023-07-06T20:14:00.067690+0000","last_unstale":"2023-07-06T20:14:01.047084+0000","last_undegraded":"2023-07-06T20:14:01.047084+0000","last_fullsized":"2023-07-06T20:14:01.047084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239754+0000","last_change":"2023-07-06T20:13:59.019472+0000","last_active":"2023-07-06T20:14:01.239754+0000","last_peered":"2023-07-06T20:14:01.239754+0000","last_clean":"2023-07-06T20:14:01.239754+0000","last_became_active":"2023-07-06T20:13:59.019160+0000","last_became_peered":"2023-07-06T20:13:59.019160+0000","last_unstale":"2023-07-06T20:14:01.239754+0000","last_undegraded":"2023-07-06T20:14:01.239754+0000","last_fullsized":"2023-07-06T20:14:01.239754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240204+0000","last_change":"2023-07-06T20:14:00.061405+0000","last_active":"2023-07-06T20:14:01.240204+0000","last_peered":"2023-07-06T20:14:01.240204+0000","last_clean":"2023-07-06T20:14:01.240204+0000","last_became_active":"2023-07-06T20:14:00.060981+0000","last_became_peered":"2023-07-06T20:14:00.060981+0000","last_unstale":"2023-07-06T20:14:01.240204+0000","last_undegraded":"2023-07-06T20:14:01.240204+0000","last_fullsized":"2023-07-06T20:14:01.240204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047136+0000","last_change":"2023-07-06T20:13:59.022367+0000","last_active":"2023-07-06T20:14:01.047136+0000","last_peered":"2023-07-06T20:14:01.047136+0000","last_clean":"2023-07-06T20:14:01.047136+0000","last_became_active":"2023-07-06T20:13:59.022105+0000","last_became_peered":"2023-07-06T20:13:59.022105+0000","last_unstale":"2023-07-06T20:14:01.047136+0000","last_undegraded":"2023-07-06T20:14:01.047136+0000","last_fullsized":"2023-07-06T20:14:01.047136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239984+0000","last_change":"2023-07-06T20:14:00.052443+0000","last_active":"2023-07-06T20:14:01.239984+0000","last_peered":"2023-07-06T20:14:01.239984+0000","last_clean":"2023-07-06T20:14:01.239984+0000","last_became_active":"2023-07-06T20:14:00.052132+0000","last_became_peered":"2023-07-06T20:14:00.052132+0000","last_unstale":"2023-07-06T20:14:01.239984+0000","last_undegraded":"2023-07-06T20:14:01.239984+0000","last_fullsized":"2023-07-06T20:14:01.239984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:24:53.471708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239579+0000","last_change":"2023-07-06T20:13:59.029867+0000","last_active":"2023-07-06T20:14:01.239579+0000","last_peered":"2023-07-06T20:14:01.239579+0000","last_clean":"2023-07-06T20:14:01.239579+0000","last_became_active":"2023-07-06T20:13:59.029567+0000","last_became_peered":"2023-07-06T20:13:59.029567+0000","last_unstale":"2023-07-06T20:14:01.239579+0000","last_undegraded":"2023-07-06T20:14:01.239579+0000","last_fullsized":"2023-07-06T20:14:01.239579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:03.269578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:14:00.067811+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:14:00.067528+0000","last_became_peered":"2023-07-06T20:14:00.067528+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:10.117201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239990+0000","last_change":"2023-07-06T20:13:59.021331+0000","last_active":"2023-07-06T20:14:01.239990+0000","last_peered":"2023-07-06T20:14:01.239990+0000","last_clean":"2023-07-06T20:14:01.239990+0000","last_became_active":"2023-07-06T20:13:59.021015+0000","last_became_peered":"2023-07-06T20:13:59.021015+0000","last_unstale":"2023-07-06T20:14:01.239990+0000","last_undegraded":"2023-07-06T20:14:01.239990+0000","last_fullsized":"2023-07-06T20:14:01.239990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046539+0000","last_change":"2023-07-06T20:14:00.060824+0000","last_active":"2023-07-06T20:14:01.046539+0000","last_peered":"2023-07-06T20:14:01.046539+0000","last_clean":"2023-07-06T20:14:01.046539+0000","last_became_active":"2023-07-06T20:14:00.059809+0000","last_became_peered":"2023-07-06T20:14:00.059809+0000","last_unstale":"2023-07-06T20:14:01.046539+0000","last_undegraded":"2023-07-06T20:14:01.046539+0000","last_fullsized":"2023-07-06T20:14:01.046539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:08:00.717171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240120+0000","last_change":"2023-07-06T20:13:59.023369+0000","last_active":"2023-07-06T20:14:01.240120+0000","last_peered":"2023-07-06T20:14:01.240120+0000","last_clean":"2023-07-06T20:14:01.240120+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.240120+0000","last_undegraded":"2023-07-06T20:14:01.240120+0000","last_fullsized":"2023-07-06T20:14:01.240120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:56.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240650+0000","last_change":"2023-07-06T20:14:00.056152+0000","last_active":"2023-07-06T20:14:01.240650+0000","last_peered":"2023-07-06T20:14:01.240650+0000","last_clean":"2023-07-06T20:14:01.240650+0000","last_became_active":"2023-07-06T20:14:00.055844+0000","last_became_peered":"2023-07-06T20:14:00.055844+0000","last_unstale":"2023-07-06T20:14:01.240650+0000","last_undegraded":"2023-07-06T20:14:01.240650+0000","last_fullsized":"2023-07-06T20:14:01.240650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047386+0000","last_change":"2023-07-06T20:13:59.028781+0000","last_active":"2023-07-06T20:14:01.047386+0000","last_peered":"2023-07-06T20:14:01.047386+0000","last_clean":"2023-07-06T20:14:01.047386+0000","last_became_active":"2023-07-06T20:13:59.028475+0000","last_became_peered":"2023-07-06T20:13:59.028475+0000","last_unstale":"2023-07-06T20:14:01.047386+0000","last_undegraded":"2023-07-06T20:14:01.047386+0000","last_fullsized":"2023-07-06T20:14:01.047386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047231+0000","last_change":"2023-07-06T20:14:00.054939+0000","last_active":"2023-07-06T20:14:01.047231+0000","last_peered":"2023-07-06T20:14:01.047231+0000","last_clean":"2023-07-06T20:14:01.047231+0000","last_became_active":"2023-07-06T20:14:00.054354+0000","last_became_peered":"2023-07-06T20:14:00.054354+0000","last_unstale":"2023-07-06T20:14:01.047231+0000","last_undegraded":"2023-07-06T20:14:01.047231+0000","last_fullsized":"2023-07-06T20:14:01.047231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240033+0000","last_change":"2023-07-06T20:13:59.025672+0000","last_active":"2023-07-06T20:14:01.240033+0000","last_peered":"2023-07-06T20:14:01.240033+0000","last_clean":"2023-07-06T20:14:01.240033+0000","last_became_active":"2023-07-06T20:13:59.025314+0000","last_became_peered":"2023-07-06T20:13:59.025314+0000","last_unstale":"2023-07-06T20:14:01.240033+0000","last_undegraded":"2023-07-06T20:14:01.240033+0000","last_fullsized":"2023-07-06T20:14:01.240033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:26.780425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046831+0000","last_change":"2023-07-06T20:14:00.056179+0000","last_active":"2023-07-06T20:14:01.046831+0000","last_peered":"2023-07-06T20:14:01.046831+0000","last_clean":"2023-07-06T20:14:01.046831+0000","last_became_active":"2023-07-06T20:14:00.055858+0000","last_became_peered":"2023-07-06T20:14:00.055858+0000","last_unstale":"2023-07-06T20:14:01.046831+0000","last_undegraded":"2023-07-06T20:14:01.046831+0000","last_fullsized":"2023-07-06T20:14:01.046831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239368+0000","last_change":"2023-07-06T20:13:59.021100+0000","last_active":"2023-07-06T20:14:01.239368+0000","last_peered":"2023-07-06T20:14:01.239368+0000","last_clean":"2023-07-06T20:14:01.239368+0000","last_became_active":"2023-07-06T20:13:59.020798+0000","last_became_peered":"2023-07-06T20:13:59.020798+0000","last_unstale":"2023-07-06T20:14:01.239368+0000","last_undegraded":"2023-07-06T20:14:01.239368+0000","last_fullsized":"2023-07-06T20:14:01.239368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:48.108641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240807+0000","last_change":"2023-07-06T20:14:00.056298+0000","last_active":"2023-07-06T20:14:01.240807+0000","last_peered":"2023-07-06T20:14:01.240807+0000","last_clean":"2023-07-06T20:14:01.240807+0000","last_became_active":"2023-07-06T20:14:00.055932+0000","last_became_peered":"2023-07-06T20:14:00.055932+0000","last_unstale":"2023-07-06T20:14:01.240807+0000","last_undegraded":"2023-07-06T20:14:01.240807+0000","last_fullsized":"2023-07-06T20:14:01.240807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239561+0000","last_change":"2023-07-06T20:13:59.025533+0000","last_active":"2023-07-06T20:14:01.239561+0000","last_peered":"2023-07-06T20:14:01.239561+0000","last_clean":"2023-07-06T20:14:01.239561+0000","last_became_active":"2023-07-06T20:13:59.025225+0000","last_became_peered":"2023-07-06T20:13:59.025225+0000","last_unstale":"2023-07-06T20:14:01.239561+0000","last_undegraded":"2023-07-06T20:14:01.239561+0000","last_fullsized":"2023-07-06T20:14:01.239561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240113+0000","last_change":"2023-07-06T20:14:00.064964+0000","last_active":"2023-07-06T20:14:01.240113+0000","last_peered":"2023-07-06T20:14:01.240113+0000","last_clean":"2023-07-06T20:14:01.240113+0000","last_became_active":"2023-07-06T20:14:00.064266+0000","last_became_peered":"2023-07-06T20:14:00.064266+0000","last_unstale":"2023-07-06T20:14:01.240113+0000","last_undegraded":"2023-07-06T20:14:01.240113+0000","last_fullsized":"2023-07-06T20:14:01.240113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239123+0000","last_change":"2023-07-06T20:13:59.021088+0000","last_active":"2023-07-06T20:14:01.239123+0000","last_peered":"2023-07-06T20:14:01.239123+0000","last_clean":"2023-07-06T20:14:01.239123+0000","last_became_active":"2023-07-06T20:13:59.018712+0000","last_became_peered":"2023-07-06T20:13:59.018712+0000","last_unstale":"2023-07-06T20:14:01.239123+0000","last_undegraded":"2023-07-06T20:14:01.239123+0000","last_fullsized":"2023-07-06T20:14:01.239123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:36.875195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047315+0000","last_change":"2023-07-06T20:14:00.061173+0000","last_active":"2023-07-06T20:14:01.047315+0000","last_peered":"2023-07-06T20:14:01.047315+0000","last_clean":"2023-07-06T20:14:01.047315+0000","last_became_active":"2023-07-06T20:14:00.060346+0000","last_became_peered":"2023-07-06T20:14:00.060346+0000","last_unstale":"2023-07-06T20:14:01.047315+0000","last_undegraded":"2023-07-06T20:14:01.047315+0000","last_fullsized":"2023-07-06T20:14:01.047315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:38.739937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239999+0000","last_change":"2023-07-06T20:13:59.025539+0000","last_active":"2023-07-06T20:14:01.239999+0000","last_peered":"2023-07-06T20:14:01.239999+0000","last_clean":"2023-07-06T20:14:01.239999+0000","last_became_active":"2023-07-06T20:13:59.025201+0000","last_became_peered":"2023-07-06T20:13:59.025201+0000","last_unstale":"2023-07-06T20:14:01.239999+0000","last_undegraded":"2023-07-06T20:14:01.239999+0000","last_fullsized":"2023-07-06T20:14:01.239999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:37.222309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239775+0000","last_change":"2023-07-06T20:14:00.061044+0000","last_active":"2023-07-06T20:14:01.239775+0000","last_peered":"2023-07-06T20:14:01.239775+0000","last_clean":"2023-07-06T20:14:01.239775+0000","last_became_active":"2023-07-06T20:14:00.060737+0000","last_became_peered":"2023-07-06T20:14:00.060737+0000","last_unstale":"2023-07-06T20:14:01.239775+0000","last_undegraded":"2023-07-06T20:14:01.239775+0000","last_fullsized":"2023-07-06T20:14:01.239775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:04.272583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240322+0000","last_change":"2023-07-06T20:13:59.026669+0000","last_active":"2023-07-06T20:14:01.240322+0000","last_peered":"2023-07-06T20:14:01.240322+0000","last_clean":"2023-07-06T20:14:01.240322+0000","last_became_active":"2023-07-06T20:13:59.026297+0000","last_became_peered":"2023-07-06T20:13:59.026297+0000","last_unstale":"2023-07-06T20:14:01.240322+0000","last_undegraded":"2023-07-06T20:14:01.240322+0000","last_fullsized":"2023-07-06T20:14:01.240322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.062318+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.062063+0000","last_became_peered":"2023-07-06T20:14:00.062063+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:03.952752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240374+0000","last_change":"2023-07-06T20:13:59.027539+0000","last_active":"2023-07-06T20:14:01.240374+0000","last_peered":"2023-07-06T20:14:01.240374+0000","last_clean":"2023-07-06T20:14:01.240374+0000","last_became_active":"2023-07-06T20:13:59.027203+0000","last_became_peered":"2023-07-06T20:13:59.027203+0000","last_unstale":"2023-07-06T20:14:01.240374+0000","last_undegraded":"2023-07-06T20:14:01.240374+0000","last_fullsized":"2023-07-06T20:14:01.240374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:11.129495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240536+0000","last_change":"2023-07-06T20:14:00.064988+0000","last_active":"2023-07-06T20:14:01.240536+0000","last_peered":"2023-07-06T20:14:01.240536+0000","last_clean":"2023-07-06T20:14:01.240536+0000","last_became_active":"2023-07-06T20:14:00.064581+0000","last_became_peered":"2023-07-06T20:14:00.064581+0000","last_unstale":"2023-07-06T20:14:01.240536+0000","last_undegraded":"2023-07-06T20:14:01.240536+0000","last_fullsized":"2023-07-06T20:14:01.240536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:10.399206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239427+0000","last_change":"2023-07-06T20:13:59.021923+0000","last_active":"2023-07-06T20:14:01.239427+0000","last_peered":"2023-07-06T20:14:01.239427+0000","last_clean":"2023-07-06T20:14:01.239427+0000","last_became_active":"2023-07-06T20:13:59.021522+0000","last_became_peered":"2023-07-06T20:13:59.021522+0000","last_unstale":"2023-07-06T20:14:01.239427+0000","last_undegraded":"2023-07-06T20:14:01.239427+0000","last_fullsized":"2023-07-06T20:14:01.239427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:18.117668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239302+0000","last_change":"2023-07-06T20:14:00.065729+0000","last_active":"2023-07-06T20:14:01.239302+0000","last_peered":"2023-07-06T20:14:01.239302+0000","last_clean":"2023-07-06T20:14:01.239302+0000","last_became_active":"2023-07-06T20:14:00.064827+0000","last_became_peered":"2023-07-06T20:14:00.064827+0000","last_unstale":"2023-07-06T20:14:01.239302+0000","last_undegraded":"2023-07-06T20:14:01.239302+0000","last_fullsized":"2023-07-06T20:14:01.239302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:21:48.022430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046583+0000","last_change":"2023-07-06T20:13:59.023330+0000","last_active":"2023-07-06T20:14:01.046583+0000","last_peered":"2023-07-06T20:14:01.046583+0000","last_clean":"2023-07-06T20:14:01.046583+0000","last_became_active":"2023-07-06T20:13:59.023036+0000","last_became_peered":"2023-07-06T20:13:59.023036+0000","last_unstale":"2023-07-06T20:14:01.046583+0000","last_undegraded":"2023-07-06T20:14:01.046583+0000","last_fullsized":"2023-07-06T20:14:01.046583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:18.188230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047260+0000","last_change":"2023-07-06T20:14:00.054542+0000","last_active":"2023-07-06T20:14:01.047260+0000","last_peered":"2023-07-06T20:14:01.047260+0000","last_clean":"2023-07-06T20:14:01.047260+0000","last_became_active":"2023-07-06T20:14:00.053987+0000","last_became_peered":"2023-07-06T20:14:00.053987+0000","last_unstale":"2023-07-06T20:14:01.047260+0000","last_undegraded":"2023-07-06T20:14:01.047260+0000","last_fullsized":"2023-07-06T20:14:01.047260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:49:19.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240427+0000","last_change":"2023-07-06T20:13:59.025900+0000","last_active":"2023-07-06T20:14:01.240427+0000","last_peered":"2023-07-06T20:14:01.240427+0000","last_clean":"2023-07-06T20:14:01.240427+0000","last_became_active":"2023-07-06T20:13:59.025626+0000","last_became_peered":"2023-07-06T20:13:59.025626+0000","last_unstale":"2023-07-06T20:14:01.240427+0000","last_undegraded":"2023-07-06T20:14:01.240427+0000","last_fullsized":"2023-07-06T20:14:01.240427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:32.725703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047708+0000","last_change":"2023-07-06T20:14:00.052728+0000","last_active":"2023-07-06T20:14:01.047708+0000","last_peered":"2023-07-06T20:14:01.047708+0000","last_clean":"2023-07-06T20:14:01.047708+0000","last_became_active":"2023-07-06T20:14:00.052437+0000","last_became_peered":"2023-07-06T20:14:00.052437+0000","last_unstale":"2023-07-06T20:14:01.047708+0000","last_undegraded":"2023-07-06T20:14:01.047708+0000","last_fullsized":"2023-07-06T20:14:01.047708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:27.783430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047897+0000","last_change":"2023-07-06T20:13:59.023166+0000","last_active":"2023-07-06T20:14:01.047897+0000","last_peered":"2023-07-06T20:14:01.047897+0000","last_clean":"2023-07-06T20:14:01.047897+0000","last_became_active":"2023-07-06T20:13:59.022793+0000","last_became_peered":"2023-07-06T20:13:59.022793+0000","last_unstale":"2023-07-06T20:14:01.047897+0000","last_undegraded":"2023-07-06T20:14:01.047897+0000","last_fullsized":"2023-07-06T20:14:01.047897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:17.935325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047777+0000","last_change":"2023-07-06T20:14:00.053789+0000","last_active":"2023-07-06T20:14:01.047777+0000","last_peered":"2023-07-06T20:14:01.047777+0000","last_clean":"2023-07-06T20:14:01.047777+0000","last_became_active":"2023-07-06T20:14:00.053148+0000","last_became_peered":"2023-07-06T20:14:00.053148+0000","last_unstale":"2023-07-06T20:14:01.047777+0000","last_undegraded":"2023-07-06T20:14:01.047777+0000","last_fullsized":"2023-07-06T20:14:01.047777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:46:57.270521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046721+0000","last_change":"2023-07-06T20:13:59.020720+0000","last_active":"2023-07-06T20:14:01.046721+0000","last_peered":"2023-07-06T20:14:01.046721+0000","last_clean":"2023-07-06T20:14:01.046721+0000","last_became_active":"2023-07-06T20:13:59.020371+0000","last_became_peered":"2023-07-06T20:13:59.020371+0000","last_unstale":"2023-07-06T20:14:01.046721+0000","last_undegraded":"2023-07-06T20:14:01.046721+0000","last_fullsized":"2023-07-06T20:14:01.046721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:54.166933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240548+0000","last_change":"2023-07-06T20:14:00.056310+0000","last_active":"2023-07-06T20:14:01.240548+0000","last_peered":"2023-07-06T20:14:01.240548+0000","last_clean":"2023-07-06T20:14:01.240548+0000","last_became_active":"2023-07-06T20:14:00.055916+0000","last_became_peered":"2023-07-06T20:14:00.055916+0000","last_unstale":"2023-07-06T20:14:01.240548+0000","last_undegraded":"2023-07-06T20:14:01.240548+0000","last_fullsized":"2023-07-06T20:14:01.240548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239879+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:14:01.239879+0000","last_peered":"2023-07-06T20:14:01.239879+0000","last_clean":"2023-07-06T20:14:01.239879+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:14:01.239879+0000","last_undegraded":"2023-07-06T20:14:01.239879+0000","last_fullsized":"2023-07-06T20:14:01.239879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:12.079947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240842+0000","last_change":"2023-07-06T20:13:59.021661+0000","last_active":"2023-07-06T20:14:01.240842+0000","last_peered":"2023-07-06T20:14:01.240842+0000","last_clean":"2023-07-06T20:14:01.240842+0000","last_became_active":"2023-07-06T20:13:59.021358+0000","last_became_peered":"2023-07-06T20:13:59.021358+0000","last_unstale":"2023-07-06T20:14:01.240842+0000","last_undegraded":"2023-07-06T20:14:01.240842+0000","last_fullsized":"2023-07-06T20:14:01.240842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:59.248364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240741+0000","last_change":"2023-07-06T20:14:00.055724+0000","last_active":"2023-07-06T20:14:01.240741+0000","last_peered":"2023-07-06T20:14:01.240741+0000","last_clean":"2023-07-06T20:14:01.240741+0000","last_became_active":"2023-07-06T20:14:00.055440+0000","last_became_peered":"2023-07-06T20:14:00.055440+0000","last_unstale":"2023-07-06T20:14:01.240741+0000","last_undegraded":"2023-07-06T20:14:01.240741+0000","last_fullsized":"2023-07-06T20:14:01.240741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:45.190806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240130+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:14:01.240130+0000","last_peered":"2023-07-06T20:14:01.240130+0000","last_clean":"2023-07-06T20:14:01.240130+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:14:01.240130+0000","last_undegraded":"2023-07-06T20:14:01.240130+0000","last_fullsized":"2023-07-06T20:14:01.240130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:50.125212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047412+0000","last_change":"2023-07-06T20:13:59.023943+0000","last_active":"2023-07-06T20:14:01.047412+0000","last_peered":"2023-07-06T20:14:01.047412+0000","last_clean":"2023-07-06T20:14:01.047412+0000","last_became_active":"2023-07-06T20:13:59.023575+0000","last_became_peered":"2023-07-06T20:13:59.023575+0000","last_unstale":"2023-07-06T20:14:01.047412+0000","last_undegraded":"2023-07-06T20:14:01.047412+0000","last_fullsized":"2023-07-06T20:14:01.047412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:17.962306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240019+0000","last_change":"2023-07-06T20:14:00.055529+0000","last_active":"2023-07-06T20:14:01.240019+0000","last_peered":"2023-07-06T20:14:01.240019+0000","last_clean":"2023-07-06T20:14:01.240019+0000","last_became_active":"2023-07-06T20:14:00.054757+0000","last_became_peered":"2023-07-06T20:14:00.054757+0000","last_unstale":"2023-07-06T20:14:01.240019+0000","last_undegraded":"2023-07-06T20:14:01.240019+0000","last_fullsized":"2023-07-06T20:14:01.240019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.238739+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:14:01.238739+0000","last_peered":"2023-07-06T20:14:01.238739+0000","last_clean":"2023-07-06T20:14:01.238739+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:14:01.238739+0000","last_undegraded":"2023-07-06T20:14:01.238739+0000","last_fullsized":"2023-07-06T20:14:01.238739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:44.989647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239367+0000","last_change":"2023-07-06T20:13:59.018558+0000","last_active":"2023-07-06T20:14:01.239367+0000","last_peered":"2023-07-06T20:14:01.239367+0000","last_clean":"2023-07-06T20:14:01.239367+0000","last_became_active":"2023-07-06T20:13:59.018205+0000","last_became_peered":"2023-07-06T20:13:59.018205+0000","last_unstale":"2023-07-06T20:14:01.239367+0000","last_undegraded":"2023-07-06T20:14:01.239367+0000","last_fullsized":"2023-07-06T20:14:01.239367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240403+0000","last_change":"2023-07-06T20:14:00.062155+0000","last_active":"2023-07-06T20:14:01.240403+0000","last_peered":"2023-07-06T20:14:01.240403+0000","last_clean":"2023-07-06T20:14:01.240403+0000","last_became_active":"2023-07-06T20:14:00.061848+0000","last_became_peered":"2023-07-06T20:14:00.061848+0000","last_unstale":"2023-07-06T20:14:01.240403+0000","last_undegraded":"2023-07-06T20:14:01.240403+0000","last_fullsized":"2023-07-06T20:14:01.240403+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239665+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:14:01.239665+0000","last_peered":"2023-07-06T20:14:01.239665+0000","last_clean":"2023-07-06T20:14:01.239665+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:14:01.239665+0000","last_undegraded":"2023-07-06T20:14:01.239665+0000","last_fullsized":"2023-07-06T20:14:01.239665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:53.676445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239234+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:14:01.239234+0000","last_peered":"2023-07-06T20:14:01.239234+0000","last_clean":"2023-07-06T20:14:01.239234+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:14:01.239234+0000","last_undegraded":"2023-07-06T20:14:01.239234+0000","last_fullsized":"2023-07-06T20:14:01.239234+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:04.461531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046254+0000","last_change":"2023-07-06T20:13:59.021468+0000","last_active":"2023-07-06T20:14:01.046254+0000","last_peered":"2023-07-06T20:14:01.046254+0000","last_clean":"2023-07-06T20:14:01.046254+0000","last_became_active":"2023-07-06T20:13:59.021164+0000","last_became_peered":"2023-07-06T20:13:59.021164+0000","last_unstale":"2023-07-06T20:14:01.046254+0000","last_undegraded":"2023-07-06T20:14:01.046254+0000","last_fullsized":"2023-07-06T20:14:01.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:23:31.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239372+0000","last_change":"2023-07-06T20:14:00.062536+0000","last_active":"2023-07-06T20:14:01.239372+0000","last_peered":"2023-07-06T20:14:01.239372+0000","last_clean":"2023-07-06T20:14:01.239372+0000","last_became_active":"2023-07-06T20:14:00.062189+0000","last_became_peered":"2023-07-06T20:14:00.062189+0000","last_unstale":"2023-07-06T20:14:01.239372+0000","last_undegraded":"2023-07-06T20:14:01.239372+0000","last_fullsized":"2023-07-06T20:14:01.239372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:07:07.175614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239383+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:14:01.239383+0000","last_peered":"2023-07-06T20:14:01.239383+0000","last_clean":"2023-07-06T20:14:01.239383+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:14:01.239383+0000","last_undegraded":"2023-07-06T20:14:01.239383+0000","last_fullsized":"2023-07-06T20:14:01.239383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:38.150080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239126+0000","last_change":"2023-07-06T20:13:59.026202+0000","last_active":"2023-07-06T20:14:01.239126+0000","last_peered":"2023-07-06T20:14:01.239126+0000","last_clean":"2023-07-06T20:14:01.239126+0000","last_became_active":"2023-07-06T20:13:59.025856+0000","last_became_peered":"2023-07-06T20:13:59.025856+0000","last_unstale":"2023-07-06T20:14:01.239126+0000","last_undegraded":"2023-07-06T20:14:01.239126+0000","last_fullsized":"2023-07-06T20:14:01.239126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:42.107407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240485+0000","last_change":"2023-07-06T20:14:00.065377+0000","last_active":"2023-07-06T20:14:01.240485+0000","last_peered":"2023-07-06T20:14:01.240485+0000","last_clean":"2023-07-06T20:14:01.240485+0000","last_became_active":"2023-07-06T20:14:00.064627+0000","last_became_peered":"2023-07-06T20:14:00.064627+0000","last_unstale":"2023-07-06T20:14:01.240485+0000","last_undegraded":"2023-07-06T20:14:01.240485+0000","last_fullsized":"2023-07-06T20:14:01.240485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.965310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240293+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:14:01.240293+0000","last_peered":"2023-07-06T20:14:01.240293+0000","last_clean":"2023-07-06T20:14:01.240293+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:14:01.240293+0000","last_undegraded":"2023-07-06T20:14:01.240293+0000","last_fullsized":"2023-07-06T20:14:01.240293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:41:59.625038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240045+0000","last_change":"2023-07-06T20:13:59.027140+0000","last_active":"2023-07-06T20:14:01.240045+0000","last_peered":"2023-07-06T20:14:01.240045+0000","last_clean":"2023-07-06T20:14:01.240045+0000","last_became_active":"2023-07-06T20:13:59.026802+0000","last_became_peered":"2023-07-06T20:13:59.026802+0000","last_unstale":"2023-07-06T20:14:01.240045+0000","last_undegraded":"2023-07-06T20:14:01.240045+0000","last_fullsized":"2023-07-06T20:14:01.240045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239757+0000","last_change":"2023-07-06T20:14:00.057341+0000","last_active":"2023-07-06T20:14:01.239757+0000","last_peered":"2023-07-06T20:14:01.239757+0000","last_clean":"2023-07-06T20:14:01.239757+0000","last_became_active":"2023-07-06T20:14:00.057077+0000","last_became_peered":"2023-07-06T20:14:00.057077+0000","last_unstale":"2023-07-06T20:14:01.239757+0000","last_undegraded":"2023-07-06T20:14:01.239757+0000","last_fullsized":"2023-07-06T20:14:01.239757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047029+0000","last_change":"2023-07-06T20:13:59.023627+0000","last_active":"2023-07-06T20:14:01.047029+0000","last_peered":"2023-07-06T20:14:01.047029+0000","last_clean":"2023-07-06T20:14:01.047029+0000","last_became_active":"2023-07-06T20:13:59.023332+0000","last_became_peered":"2023-07-06T20:13:59.023332+0000","last_unstale":"2023-07-06T20:14:01.047029+0000","last_undegraded":"2023-07-06T20:14:01.047029+0000","last_fullsized":"2023-07-06T20:14:01.047029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:37.193187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240442+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:14:01.240442+0000","last_peered":"2023-07-06T20:14:01.240442+0000","last_clean":"2023-07-06T20:14:01.240442+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:14:01.240442+0000","last_undegraded":"2023-07-06T20:14:01.240442+0000","last_fullsized":"2023-07-06T20:14:01.240442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:03.076475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046144+0000","last_change":"2023-07-06T20:14:00.066131+0000","last_active":"2023-07-06T20:14:01.046144+0000","last_peered":"2023-07-06T20:14:01.046144+0000","last_clean":"2023-07-06T20:14:01.046144+0000","last_became_active":"2023-07-06T20:14:00.065793+0000","last_became_peered":"2023-07-06T20:14:00.065793+0000","last_unstale":"2023-07-06T20:14:01.046144+0000","last_undegraded":"2023-07-06T20:14:01.046144+0000","last_fullsized":"2023-07-06T20:14:01.046144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:49.111646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239813+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:14:01.239813+0000","last_peered":"2023-07-06T20:14:01.239813+0000","last_clean":"2023-07-06T20:14:01.239813+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:14:01.239813+0000","last_undegraded":"2023-07-06T20:14:01.239813+0000","last_fullsized":"2023-07-06T20:14:01.239813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:17.841824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240384+0000","last_change":"2023-07-06T20:13:59.023885+0000","last_active":"2023-07-06T20:14:01.240384+0000","last_peered":"2023-07-06T20:14:01.240384+0000","last_clean":"2023-07-06T20:14:01.240384+0000","last_became_active":"2023-07-06T20:13:59.023631+0000","last_became_peered":"2023-07-06T20:13:59.023631+0000","last_unstale":"2023-07-06T20:14:01.240384+0000","last_undegraded":"2023-07-06T20:14:01.240384+0000","last_fullsized":"2023-07-06T20:14:01.240384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047806+0000","last_change":"2023-07-06T20:14:00.055463+0000","last_active":"2023-07-06T20:14:01.047806+0000","last_peered":"2023-07-06T20:14:01.047806+0000","last_clean":"2023-07-06T20:14:01.047806+0000","last_became_active":"2023-07-06T20:14:00.055196+0000","last_became_peered":"2023-07-06T20:14:00.055196+0000","last_unstale":"2023-07-06T20:14:01.047806+0000","last_undegraded":"2023-07-06T20:14:01.047806+0000","last_fullsized":"2023-07-06T20:14:01.047806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:42:06.665764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239438+0000","last_change":"2023-07-06T20:13:59.026503+0000","last_active":"2023-07-06T20:14:01.239438+0000","last_peered":"2023-07-06T20:14:01.239438+0000","last_clean":"2023-07-06T20:14:01.239438+0000","last_became_active":"2023-07-06T20:13:59.026265+0000","last_became_peered":"2023-07-06T20:13:59.026265+0000","last_unstale":"2023-07-06T20:14:01.239438+0000","last_undegraded":"2023-07-06T20:14:01.239438+0000","last_fullsized":"2023-07-06T20:14:01.239438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239401+0000","last_change":"2023-07-06T20:14:00.067006+0000","last_active":"2023-07-06T20:14:01.239401+0000","last_peered":"2023-07-06T20:14:01.239401+0000","last_clean":"2023-07-06T20:14:01.239401+0000","last_became_active":"2023-07-06T20:14:00.066735+0000","last_became_peered":"2023-07-06T20:14:00.066735+0000","last_unstale":"2023-07-06T20:14:01.239401+0000","last_undegraded":"2023-07-06T20:14:01.239401+0000","last_fullsized":"2023-07-06T20:14:01.239401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:00:37.878199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239281+0000","last_change":"2023-07-06T20:13:59.024148+0000","last_active":"2023-07-06T20:14:01.239281+0000","last_peered":"2023-07-06T20:14:01.239281+0000","last_clean":"2023-07-06T20:14:01.239281+0000","last_became_active":"2023-07-06T20:13:59.023838+0000","last_became_peered":"2023-07-06T20:13:59.023838+0000","last_unstale":"2023-07-06T20:14:01.239281+0000","last_undegraded":"2023-07-06T20:14:01.239281+0000","last_fullsized":"2023-07-06T20:14:01.239281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:23.609457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047642+0000","last_change":"2023-07-06T20:14:00.054948+0000","last_active":"2023-07-06T20:14:01.047642+0000","last_peered":"2023-07-06T20:14:01.047642+0000","last_clean":"2023-07-06T20:14:01.047642+0000","last_became_active":"2023-07-06T20:14:00.054677+0000","last_became_peered":"2023-07-06T20:14:00.054677+0000","last_unstale":"2023-07-06T20:14:01.047642+0000","last_undegraded":"2023-07-06T20:14:01.047642+0000","last_fullsized":"2023-07-06T20:14:01.047642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:51:38.225314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.240958+0000","last_change":"2023-07-06T20:13:59.023297+0000","last_active":"2023-07-06T20:14:01.240958+0000","last_peered":"2023-07-06T20:14:01.240958+0000","last_clean":"2023-07-06T20:14:01.240958+0000","last_became_active":"2023-07-06T20:13:59.023023+0000","last_became_peered":"2023-07-06T20:13:59.023023+0000","last_unstale":"2023-07-06T20:14:01.240958+0000","last_undegraded":"2023-07-06T20:14:01.240958+0000","last_fullsized":"2023-07-06T20:14:01.240958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:37.736933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047478+0000","last_change":"2023-07-06T20:14:00.053372+0000","last_active":"2023-07-06T20:14:01.047478+0000","last_peered":"2023-07-06T20:14:01.047478+0000","last_clean":"2023-07-06T20:14:01.047478+0000","last_became_active":"2023-07-06T20:14:00.053043+0000","last_became_peered":"2023-07-06T20:14:00.053043+0000","last_unstale":"2023-07-06T20:14:01.047478+0000","last_undegraded":"2023-07-06T20:14:01.047478+0000","last_fullsized":"2023-07-06T20:14:01.047478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:38.196192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046788+0000","last_change":"2023-07-06T20:13:59.023910+0000","last_active":"2023-07-06T20:14:01.046788+0000","last_peered":"2023-07-06T20:14:01.046788+0000","last_clean":"2023-07-06T20:14:01.046788+0000","last_became_active":"2023-07-06T20:13:59.023622+0000","last_became_peered":"2023-07-06T20:13:59.023622+0000","last_unstale":"2023-07-06T20:14:01.046788+0000","last_undegraded":"2023-07-06T20:14:01.046788+0000","last_fullsized":"2023-07-06T20:14:01.046788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:14:56.162933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046413+0000","last_change":"2023-07-06T20:14:00.038980+0000","last_active":"2023-07-06T20:14:01.046413+0000","last_peered":"2023-07-06T20:14:01.046413+0000","last_clean":"2023-07-06T20:14:01.046413+0000","last_became_active":"2023-07-06T20:14:00.038706+0000","last_became_peered":"2023-07-06T20:14:00.038706+0000","last_unstale":"2023-07-06T20:14:01.046413+0000","last_undegraded":"2023-07-06T20:14:01.046413+0000","last_fullsized":"2023-07-06T20:14:01.046413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:54:19.120673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239873+0000","last_change":"2023-07-06T20:13:59.029660+0000","last_active":"2023-07-06T20:14:01.239873+0000","last_peered":"2023-07-06T20:14:01.239873+0000","last_clean":"2023-07-06T20:14:01.239873+0000","last_became_active":"2023-07-06T20:13:59.029408+0000","last_became_peered":"2023-07-06T20:13:59.029408+0000","last_unstale":"2023-07-06T20:14:01.239873+0000","last_undegraded":"2023-07-06T20:14:01.239873+0000","last_fullsized":"2023-07-06T20:14:01.239873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:57.992861+0000","last_clean_scrub_stamp":"2023-07-06T20:13:57.992861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:59.714166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047021+0000","last_change":"2023-07-06T20:14:00.049540+0000","last_active":"2023-07-06T20:14:01.047021+0000","last_peered":"2023-07-06T20:14:01.047021+0000","last_clean":"2023-07-06T20:14:01.047021+0000","last_became_active":"2023-07-06T20:14:00.049260+0000","last_became_peered":"2023-07-06T20:14:00.049260+0000","last_unstale":"2023-07-06T20:14:01.047021+0000","last_undegraded":"2023-07-06T20:14:01.047021+0000","last_fullsized":"2023-07-06T20:14:01.047021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239288+0000","last_change":"2023-07-06T20:14:00.048736+0000","last_active":"2023-07-06T20:14:01.239288+0000","last_peered":"2023-07-06T20:14:01.239288+0000","last_clean":"2023-07-06T20:14:01.239288+0000","last_became_active":"2023-07-06T20:14:00.048418+0000","last_became_peered":"2023-07-06T20:14:00.048418+0000","last_unstale":"2023-07-06T20:14:01.239288+0000","last_undegraded":"2023-07-06T20:14:01.239288+0000","last_fullsized":"2023-07-06T20:14:01.239288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:28:20.663674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046375+0000","last_change":"2023-07-06T20:14:00.067909+0000","last_active":"2023-07-06T20:14:01.046375+0000","last_peered":"2023-07-06T20:14:01.046375+0000","last_clean":"2023-07-06T20:14:01.046375+0000","last_became_active":"2023-07-06T20:14:00.067600+0000","last_became_peered":"2023-07-06T20:14:00.067600+0000","last_unstale":"2023-07-06T20:14:01.046375+0000","last_undegraded":"2023-07-06T20:14:01.046375+0000","last_fullsized":"2023-07-06T20:14:01.046375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.047258+0000","last_change":"2023-07-06T20:14:00.058103+0000","last_active":"2023-07-06T20:14:01.047258+0000","last_peered":"2023-07-06T20:14:01.047258+0000","last_clean":"2023-07-06T20:14:01.047258+0000","last_became_active":"2023-07-06T20:14:00.057800+0000","last_became_peered":"2023-07-06T20:14:00.057800+0000","last_unstale":"2023-07-06T20:14:01.047258+0000","last_undegraded":"2023-07-06T20:14:01.047258+0000","last_fullsized":"2023-07-06T20:14:01.047258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:55.169937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.046528+0000","last_change":"2023-07-06T20:14:00.061079+0000","last_active":"2023-07-06T20:14:01.046528+0000","last_peered":"2023-07-06T20:14:01.046528+0000","last_clean":"2023-07-06T20:14:01.046528+0000","last_became_active":"2023-07-06T20:14:00.059768+0000","last_became_peered":"2023-07-06T20:14:00.059768+0000","last_unstale":"2023-07-06T20:14:01.046528+0000","last_undegraded":"2023-07-06T20:14:01.046528+0000","last_fullsized":"2023-07-06T20:14:01.046528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:09.284613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T20:14:01.239855+0000","last_change":"2023-07-06T20:14:00.056153+0000","last_active":"2023-07-06T20:14:01.239855+0000","last_peered":"2023-07-06T20:14:01.239855+0000","last_clean":"2023-07-06T20:14:01.239855+0000","last_became_active":"2023-07-06T20:14:00.055830+0000","last_became_peered":"2023-07-06T20:14:00.055830+0000","last_unstale":"2023-07-06T20:14:01.239855+0000","last_undegraded":"2023-07-06T20:14:01.239855+0000","last_fullsized":"2023-07-06T20:14:01.239855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:58.995865+0000","last_clean_scrub_stamp":"2023-07-06T20:13:58.995865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:43:24.612461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.665}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.208}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.997999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.247999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.0010000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7939999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1710000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:14:15.398 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T20:14:15.398 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-06T20:14:15.399 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-06T20:14:15.399 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T20:14:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.629+0000 7f24eb7ff700 1 Processor -- start 2023-07-06T20:14:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.630+0000 7f24eb7ff700 1 -- start start 2023-07-06T20:14:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24eb7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e406c330 0x7f24e406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24eb7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24e40c9480 0x7f24e4156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24eb7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e406cc40 con 0x7f24e40c9010 2023-07-06T20:14:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24eb7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40c9a60 con 0x7f24e40c9480 2023-07-06T20:14:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24eb7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4159b70 con 0x7f24e406c330 2023-07-06T20:14:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24e9d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24e40c9480 0x7f24e4156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24e9d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24e40c9480 0x7f24e4156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44930/0 (socket says 172.21.15.27:44930) 2023-07-06T20:14:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24e9d9c700 1 -- 172.21.15.27:0/3271721033 learned_addr learned my addr 172.21.15.27:0/3271721033 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:15.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.631+0000 7f24e959b700 1 --2- 172.21.15.27:0/3271721033 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e406c330 0x7f24e406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24e9d9c700 1 -- 172.21.15.27:0/3271721033 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e406c330 msgr2=0x7f24e406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24e9d9c700 1 --2- 172.21.15.27:0/3271721033 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e406c330 0x7f24e406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24e9d9c700 1 -- 172.21.15.27:0/3271721033 >> v1:172.21.15.153:6789/0 conn(0x7f24e40c9010 legacy=0x7f24e41565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24e9d9c700 1 -- 172.21.15.27:0/3271721033 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4159e50 con 0x7f24e40c9480 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24e9d9c700 1 --2- 172.21.15.27:0/3271721033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e40c9480 0x7f24e4156d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009e50 tx=0x7f24dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6819a14734f1a68c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24dbfff700 1 -- 172.21.15.27:0/3271721033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00e070 con 0x7f24e40c9480 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24dbfff700 1 -- 172.21.15.27:0/3271721033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc00b9c0 con 0x7f24e40c9480 2023-07-06T20:14:15.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.632+0000 7f24dbfff700 1 -- 172.21.15.27:0/3271721033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00bd70 con 0x7f24e40c9480 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 -- 172.21.15.27:0/3271721033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e40c9480 msgr2=0x7f24e4156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/3271721033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e40c9480 0x7f24e4156d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009e50 tx=0x7f24dc00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 -- 172.21.15.27:0/3271721033 shutdown_connections 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/3271721033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e40c9480 0x7f24e4156d00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/3271721033 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e406c330 0x7f24e406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 -- 172.21.15.27:0/3271721033 >> 172.21.15.27:0/3271721033 conn(0x7f24e40cb440 msgr2=0x7f24e40cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 -- 172.21.15.27:0/3271721033 shutdown_connections 2023-07-06T20:14:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.633+0000 7f24eb7ff700 1 -- 172.21.15.27:0/3271721033 wait complete. 2023-07-06T20:14:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.635+0000 7f24eb7ff700 1 Processor -- start 2023-07-06T20:14:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.635+0000 7f24eb7ff700 1 -- start start 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24eb7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 0x7f24e40c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24eb7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e40c9010 0x7f24e40acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24eb7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e40c9480 0x7f24e414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24eb7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40c40b0 con 0x7f24e406c330 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24eb7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e414df90 con 0x7f24e40c9010 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24eb7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40c4300 con 0x7f24e40c9480 2023-07-06T20:14:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 0x7f24e40c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 0x7f24e40c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44938/0 (socket says 172.21.15.27:44938) 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 -- 172.21.15.27:0/4258448668 learned_addr learned my addr 172.21.15.27:0/4258448668 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e9d9c700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e40c9480 0x7f24e414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e8d9a700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e40c9010 0x7f24e40acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 -- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e40c9480 msgr2=0x7f24e414ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e40c9480 0x7f24e414ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 -- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e40c9010 msgr2=0x7f24e40acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e40c9010 0x7f24e40acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.636+0000 7f24e959b700 1 -- 172.21.15.27:0/4258448668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc009b30 con 0x7f24e406c330 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.637+0000 7f24e959b700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 0x7f24e40c4de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d800 tx=0x7f24e000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.637+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00125f0 con 0x7f24e406c330 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.637+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0012750 con 0x7f24e406c330 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.637+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e40c45e0 con 0x7f24e406c330 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.637+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0012b00 con 0x7f24e406c330 2023-07-06T20:14:15.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.637+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e40c5320 con 0x7f24e406c330 2023-07-06T20:14:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.638+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40c4c30 con 0x7f24e406c330 2023-07-06T20:14:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.639+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24e0012c60 con 0x7f24e406c330 2023-07-06T20:14:15.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.640+0000 7f24d9ffb700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f24c806ace0 0x7f24c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:15.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.640+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24e0089560 con 0x7f24e406c330 2023-07-06T20:14:15.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.641+0000 7f24e8d9a700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f24c806ace0 0x7f24c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:15.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.641+0000 7f24e8d9a700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f24c806ace0 0x7f24c806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24e40c2040 tx=0x7f24d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:15.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.644+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f24e0054570 con 0x7f24e406c330 2023-07-06T20:14:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.897+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f24e40008d0 con 0x7f24e406c330 2023-07-06T20:14:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.897+0000 7f24d9ffb700 1 -- 172.21.15.27:0/4258448668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f24e0057b90 con 0x7f24e406c330 2023-07-06T20:14:15.900 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:15.900 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T20:14:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f24c806ace0 msgr2=0x7f24c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f24c806ace0 0x7f24c806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24e40c2040 tx=0x7f24d4009040 comp rx=0 tx=0).stop 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 msgr2=0x7f24e40c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 0x7f24e40c4de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d800 tx=0x7f24e000db10 comp rx=0 tx=0).stop 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 shutdown_connections 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f24c806ace0 0x7f24c806d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24e40c9480 0x7f24e414ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e406c330 0x7f24e40c4de0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 --2- 172.21.15.27:0/4258448668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24e40c9010 0x7f24e40acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 >> 172.21.15.27:0/4258448668 conn(0x7f24e40cb440 msgr2=0x7f24e41585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 shutdown_connections 2023-07-06T20:14:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:15.900+0000 7f24eb7ff700 1 -- 172.21.15.27:0/4258448668 wait complete. 2023-07-06T20:14:15.919 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-06T20:14:15.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:16.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.129+0000 7fc25f1ff700 1 Processor -- start 2023-07-06T20:14:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.129+0000 7fc25f1ff700 1 -- start start 2023-07-06T20:14:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.130+0000 7fc25f1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc25815fca0 0x7fc2580d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc25f1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2580cf780 con 0x7fc2580b6100 2023-07-06T20:14:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc25f1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2580cd490 con 0x7fc2580b5c90 2023-07-06T20:14:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc25f1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580cd770 con 0x7fc25815fca0 2023-07-06T20:14:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc25d79c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc2580b6100 0x7fc2580cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36094/0 (socket says 172.21.15.27:36094) 2023-07-06T20:14:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc25d79c700 1 -- 172.21.15.27:0/2019961595 learned_addr learned my addr 172.21.15.27:0/2019961595 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc25cf9b700 1 --2- 172.21.15.27:0/2019961595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc25815fca0 0x7fc2580d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372432342 0 0) 0x7fc2580cf780 con 0x7fc2580b6100 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2480035b0 con 0x7fc2580b6100 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432097063 0 0) 0x7fc2580cd490 con 0x7fc2580b5c90 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.131+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2580cf780 con 0x7fc2580b5c90 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc25cf9b700 1 -- 172.21.15.27:0/2019961595 >> v1:172.21.15.153:6789/0 conn(0x7fc2580b5c90 legacy=0x7fc2580d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc25cf9b700 1 -- 172.21.15.27:0/2019961595 >> v1:172.21.15.27:6789/0 conn(0x7fc2580b6100 legacy=0x7fc2580cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc25cf9b700 1 -- 172.21.15.27:0/2019961595 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580cda50 con 0x7fc25815fca0 2023-07-06T20:14:16.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc25cf9b700 1 --2- 172.21.15.27:0/2019961595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc25815fca0 0x7fc2580d1420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009870 tx=0x7fc24c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab17eb08b23206a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c011830 con 0x7fc25815fca0 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c011990 con 0x7fc25815fca0 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.132+0000 7fc2577fe700 1 -- 172.21.15.27:0/2019961595 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c011d20 con 0x7fc25815fca0 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2019961595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc25815fca0 msgr2=0x7fc2580d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2019961595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc25815fca0 0x7fc2580d1420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009870 tx=0x7fc24c009b80 comp rx=0 tx=0).stop 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2019961595 shutdown_connections 2023-07-06T20:14:16.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2019961595 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc25815fca0 0x7fc2580d1420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2019961595 >> 172.21.15.27:0/2019961595 conn(0x7fc25806c150 msgr2=0x7fc25806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:16.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2019961595 shutdown_connections 2023-07-06T20:14:16.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.133+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2019961595 wait complete. 2023-07-06T20:14:16.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.135+0000 7fc25f1ff700 1 Processor -- start 2023-07-06T20:14:16.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.135+0000 7fc25f1ff700 1 -- start start 2023-07-06T20:14:16.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25f1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2580b5c90 0x7fc2581f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25f1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2580b6100 0x7fc2580cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25f1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 0x7fc2580b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25f1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2581f1d50 con 0x7fc25815fca0 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25f1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2581f2030 con 0x7fc2580b6100 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25f1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2581f6270 con 0x7fc2580b5c90 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 0x7fc2580b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 0x7fc2580b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44944/0 (socket says 172.21.15.27:44944) 2023-07-06T20:14:16.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 -- 172.21.15.27:0/2744347286 learned_addr learned my addr 172.21.15.27:0/2744347286 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25cf9b700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2580b5c90 0x7fc2581f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc257fff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2580b6100 0x7fc2580cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 -- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2580b5c90 msgr2=0x7fc2581f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2580b5c90 0x7fc2581f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 -- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2580b6100 msgr2=0x7fc2580cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2580b6100 0x7fc2580cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.136+0000 7fc25d79c700 1 -- 172.21.15.27:0/2744347286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0094a0 con 0x7fc25815fca0 2023-07-06T20:14:16.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc25cf9b700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2580b5c90 0x7fc2581f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:16.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc25d79c700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 0x7fc2580b37b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc25000cbc0 tx=0x7fc25000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:16.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000ab10 con 0x7fc25815fca0 2023-07-06T20:14:16.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25000ac70 con 0x7fc25815fca0 2023-07-06T20:14:16.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2581f59d0 con 0x7fc25815fca0 2023-07-06T20:14:16.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250012400 con 0x7fc25815fca0 2023-07-06T20:14:16.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.137+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2581f5f90 con 0x7fc25815fca0 2023-07-06T20:14:16.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.138+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25815e7b0 con 0x7fc25815fca0 2023-07-06T20:14:16.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.139+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc25001b3b0 con 0x7fc25815fca0 2023-07-06T20:14:16.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.140+0000 7fc2557fa700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc23c06ace0 0x7fc23c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.140+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2500892e0 con 0x7fc25815fca0 2023-07-06T20:14:16.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.141+0000 7fc25cf9b700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc23c06ace0 0x7fc23c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.142+0000 7fc25cf9b700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc23c06ace0 0x7fc23c06d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0090b0 tx=0x7fc24c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:16.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.145+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc250054380 con 0x7fc25815fca0 2023-07-06T20:14:16.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.353+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2580008d0 con 0x7fc25815fca0 2023-07-06T20:14:16.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.354+0000 7fc2557fa700 1 -- 172.21.15.27:0/2744347286 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7fc2500579a0 con 0x7fc25815fca0 2023-07-06T20:14:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:16.357 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc23c06ace0 msgr2=0x7fc23c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc23c06ace0 0x7fc23c06d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0090b0 tx=0x7fc24c01c040 comp rx=0 tx=0).stop 2023-07-06T20:14:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 msgr2=0x7fc2580b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 0x7fc2580b37b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc25000cbc0 tx=0x7fc25000cf80 comp rx=0 tx=0).stop 2023-07-06T20:14:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 shutdown_connections 2023-07-06T20:14:16.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc23c06ace0 0x7fc23c06d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc25815fca0 0x7fc2580b37b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc2580b5c90 0x7fc2581f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 --2- 172.21.15.27:0/2744347286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2580b6100 0x7fc2580cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 >> 172.21.15.27:0/2744347286 conn(0x7fc25806c150 msgr2=0x7fc2580b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:16.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 shutdown_connections 2023-07-06T20:14:16.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.357+0000 7fc25f1ff700 1 -- 172.21.15.27:0/2744347286 wait complete. 2023-07-06T20:14:16.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:16.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.582+0000 7fe6629ff700 1 Processor -- start 2023-07-06T20:14:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.582+0000 7fe6629ff700 1 -- start start 2023-07-06T20:14:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.583+0000 7fe6629ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe65c15fca0 0x7fe65c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.583+0000 7fe6629ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe65c0b6110 0x7fe65c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.583+0000 7fe6629ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65c0cf780 con 0x7fe65c0b5c90 2023-07-06T20:14:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.583+0000 7fe6629ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c0cd490 con 0x7fe65c15fca0 2023-07-06T20:14:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.583+0000 7fe6629ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c0cd770 con 0x7fe65c0b6110 2023-07-06T20:14:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe65c15fca0 0x7fe65c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe65c15fca0 0x7fe65c0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44954/0 (socket says 172.21.15.27:44954) 2023-07-06T20:14:16.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 -- 172.21.15.27:0/4117971899 learned_addr learned my addr 172.21.15.27:0/4117971899 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:16.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65b7fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe65c0b5c90 0x7fe65c0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36098/0 (socket says 172.21.15.27:36098) 2023-07-06T20:14:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe660f9c700 1 --2- 172.21.15.27:0/4117971899 >> v2:172.21.15.153:3300/0 conn(0x7fe65c0b6110 0x7fe65c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 -- 172.21.15.27:0/4117971899 >> v2:172.21.15.153:3300/0 conn(0x7fe65c0b6110 msgr2=0x7fe65c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 --2- 172.21.15.27:0/4117971899 >> v2:172.21.15.153:3300/0 conn(0x7fe65c0b6110 0x7fe65c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 -- 172.21.15.27:0/4117971899 >> v1:172.21.15.27:6789/0 conn(0x7fe65c0b5c90 legacy=0x7fe65c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 -- 172.21.15.27:0/4117971899 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0cda50 con 0x7fe65c15fca0 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe660f9c700 1 --2- 172.21.15.27:0/4117971899 >> v2:172.21.15.153:3300/0 conn(0x7fe65c0b6110 0x7fe65c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.584+0000 7fe65bfff700 1 --2- 172.21.15.27:0/4117971899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c15fca0 0x7fe65c0d1420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0cd410 tx=0x7fe64c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6796f1292f0b635a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe65affd700 1 -- 172.21.15.27:0/4117971899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00b870 con 0x7fe65c15fca0 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe65affd700 1 -- 172.21.15.27:0/4117971899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c00b9d0 con 0x7fe65c15fca0 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe65affd700 1 -- 172.21.15.27:0/4117971899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00bd60 con 0x7fe65c15fca0 2023-07-06T20:14:16.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 -- 172.21.15.27:0/4117971899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c15fca0 msgr2=0x7fe65c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 --2- 172.21.15.27:0/4117971899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c15fca0 0x7fe65c0d1420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0cd410 tx=0x7fe64c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 -- 172.21.15.27:0/4117971899 shutdown_connections 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 --2- 172.21.15.27:0/4117971899 >> v2:172.21.15.153:3300/0 conn(0x7fe65c0b6110 0x7fe65c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 --2- 172.21.15.27:0/4117971899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c15fca0 0x7fe65c0d1420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 -- 172.21.15.27:0/4117971899 >> 172.21.15.27:0/4117971899 conn(0x7fe65c06c150 msgr2=0x7fe65c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.585+0000 7fe6629ff700 1 -- 172.21.15.27:0/4117971899 shutdown_connections 2023-07-06T20:14:16.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.586+0000 7fe6629ff700 1 -- 172.21.15.27:0/4117971899 wait complete. 2023-07-06T20:14:16.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.587+0000 7fe6629ff700 1 Processor -- start 2023-07-06T20:14:16.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.588+0000 7fe6629ff700 1 -- start start 2023-07-06T20:14:16.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe6629ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 0x7fe65c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe6629ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe65c0b6110 0x7fe65c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe6629ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe65c15fca0 0x7fe65c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe6629ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0c7650 con 0x7fe65c0b5c90 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe6629ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0c69d0 con 0x7fe65c0b6110 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe6629ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0c6cb0 con 0x7fe65c15fca0 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 0x7fe65c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 0x7fe65c14d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44962/0 (socket says 172.21.15.27:44962) 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 -- 172.21.15.27:0/2673081512 learned_addr learned my addr 172.21.15.27:0/2673081512 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65b7fe700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe65c0b6110 0x7fe65c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe660f9c700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe65c15fca0 0x7fe65c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 -- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe65c15fca0 msgr2=0x7fe65c0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe65c15fca0 0x7fe65c0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 -- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe65c0b6110 msgr2=0x7fe65c0c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe65c0b6110 0x7fe65c0c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe65bfff700 1 -- 172.21.15.27:0/2673081512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c009550 con 0x7fe65c0b5c90 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.589+0000 7fe660f9c700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe65c15fca0 0x7fe65c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.590+0000 7fe65bfff700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 0x7fe65c14d210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe64c014040 tx=0x7fe64c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.590+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0128c0 con 0x7fe65c0b5c90 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.590+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c01e070 con 0x7fe65c0b5c90 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.590+0000 7fe6629ff700 1 -- 172.21.15.27:0/2673081512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c0c6f90 con 0x7fe65c0b5c90 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.590+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c012c60 con 0x7fe65c0b5c90 2023-07-06T20:14:16.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.590+0000 7fe6629ff700 1 -- 172.21.15.27:0/2673081512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c14d750 con 0x7fe65c0b5c90 2023-07-06T20:14:16.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.591+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6340052f0 con 0x7fe65c0b5c90 2023-07-06T20:14:16.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.592+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe64c02d450 con 0x7fe65c0b5c90 2023-07-06T20:14:16.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.593+0000 7fe658ff9700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe64806ace0 0x7fe64806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:16.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.593+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe64c08a670 con 0x7fe65c0b5c90 2023-07-06T20:14:16.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.593+0000 7fe65b7fe700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe64806ace0 0x7fe64806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:16.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.594+0000 7fe65b7fe700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe64806ace0 0x7fe64806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe65c1f68d0 tx=0x7fe650007f10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:16.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.597+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe64c055690 con 0x7fe65c0b5c90 2023-07-06T20:14:16.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.804+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe634005b20 con 0x7fe65c0b5c90 2023-07-06T20:14:16.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.804+0000 7fe658ff9700 1 -- 172.21.15.27:0/2673081512 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe64c058cb0 con 0x7fe65c0b5c90 2023-07-06T20:14:16.806 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:16.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe64806ace0 msgr2=0x7fe64806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe64806ace0 0x7fe64806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe65c1f68d0 tx=0x7fe650007f10 comp rx=0 tx=0).stop 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 msgr2=0x7fe65c14d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 0x7fe65c14d210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe64c014040 tx=0x7fe64c004750 comp rx=0 tx=0).stop 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 shutdown_connections 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe64806ace0 0x7fe64806d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe65c15fca0 0x7fe65c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe65c0b5c90 0x7fe65c14d210 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 --2- 172.21.15.27:0/2673081512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe65c0b6110 0x7fe65c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:16.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.806+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 >> 172.21.15.27:0/2673081512 conn(0x7fe65c06c150 msgr2=0x7fe65c15a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:16.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.807+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 shutdown_connections 2023-07-06T20:14:16.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:16.807+0000 7fe6427fc700 1 -- 172.21.15.27:0/2673081512 wait complete. 2023-07-06T20:14:16.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:17.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.032+0000 7f83be1ff700 1 Processor -- start 2023-07-06T20:14:17.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.033+0000 7f83be1ff700 1 -- start start 2023-07-06T20:14:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83be1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83b815fca0 0x7f83b80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83be1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b80cf780 con 0x7f83b80b5c90 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83be1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b80cd490 con 0x7f83b80b6110 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83be1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b80cd770 con 0x7f83b815fca0 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b77fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83b815fca0 0x7f83b80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b77fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83b815fca0 0x7f83b80d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44972/0 (socket says 172.21.15.27:44972) 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b6ffd700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f83b80b5c90 0x7f83b80d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36104/0 (socket says 172.21.15.27:36104) 2023-07-06T20:14:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b77fe700 1 -- 172.21.15.27:0/3807705236 learned_addr learned my addr 172.21.15.27:0/3807705236 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b77fe700 1 -- 172.21.15.27:0/3807705236 >> v1:172.21.15.153:6789/0 conn(0x7f83b80b6110 legacy=0x7f83b80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b77fe700 1 -- 172.21.15.27:0/3807705236 >> v1:172.21.15.27:6789/0 conn(0x7f83b80b5c90 legacy=0x7f83b80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b67fc700 1 -- 172.21.15.27:0/3807705236 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853078052 0 0) 0x7f83b80cf780 con 0x7f83b80b5c90 2023-07-06T20:14:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.034+0000 7f83b77fe700 1 -- 172.21.15.27:0/3807705236 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80cda50 con 0x7f83b815fca0 2023-07-06T20:14:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.035+0000 7f83b77fe700 1 --2- 172.21.15.27:0/3807705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b815fca0 0x7f83b80d1420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f83a80098f0 tx=0x7f83a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da9b7674bea50ae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.035+0000 7f83b67fc700 1 -- 172.21.15.27:0/3807705236 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800b8a0 con 0x7f83b815fca0 2023-07-06T20:14:17.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.035+0000 7f83b67fc700 1 -- 172.21.15.27:0/3807705236 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a800ba00 con 0x7f83b815fca0 2023-07-06T20:14:17.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.035+0000 7f83b67fc700 1 -- 172.21.15.27:0/3807705236 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800bd90 con 0x7f83b815fca0 2023-07-06T20:14:17.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.035+0000 7f83be1ff700 1 -- 172.21.15.27:0/3807705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b815fca0 msgr2=0x7f83b80d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.035+0000 7f83be1ff700 1 --2- 172.21.15.27:0/3807705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b815fca0 0x7f83b80d1420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f83a80098f0 tx=0x7f83a8009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.036+0000 7f83be1ff700 1 -- 172.21.15.27:0/3807705236 shutdown_connections 2023-07-06T20:14:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.036+0000 7f83be1ff700 1 --2- 172.21.15.27:0/3807705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b815fca0 0x7f83b80d1420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.036+0000 7f83be1ff700 1 -- 172.21.15.27:0/3807705236 >> 172.21.15.27:0/3807705236 conn(0x7f83b806c150 msgr2=0x7f83b806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.036+0000 7f83be1ff700 1 -- 172.21.15.27:0/3807705236 shutdown_connections 2023-07-06T20:14:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.036+0000 7f83be1ff700 1 -- 172.21.15.27:0/3807705236 wait complete. 2023-07-06T20:14:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.037+0000 7f83be1ff700 1 Processor -- start 2023-07-06T20:14:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.038+0000 7f83be1ff700 1 -- start start 2023-07-06T20:14:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.038+0000 7f83be1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 0x7f83b815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.038+0000 7f83be1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83b80b6110 0x7f83b80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.038+0000 7f83be1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b815fca0 0x7f83b814a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.038+0000 7f83be1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b814a660 con 0x7f83b80b5c90 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83be1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80caa50 con 0x7f83b815fca0 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83be1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80cad30 con 0x7f83b80b6110 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 0x7f83b815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 0x7f83b815ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44988/0 (socket says 172.21.15.27:44988) 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 -- 172.21.15.27:0/820892330 learned_addr learned my addr 172.21.15.27:0/820892330 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 -- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83b80b6110 msgr2=0x7f83b80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83b80b6110 0x7f83b80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 -- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b815fca0 msgr2=0x7f83b814a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b815fca0 0x7f83b814a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 -- 172.21.15.27:0/820892330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a8009550 con 0x7f83b80b5c90 2023-07-06T20:14:17.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f83b77fe700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 0x7f83b815ad80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83a8014040 tx=0x7f83a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.039+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80038a0 con 0x7f83b80b5c90 2023-07-06T20:14:17.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.040+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a801e070 con 0x7f83b80b5c90 2023-07-06T20:14:17.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.040+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b80cb010 con 0x7f83b80b5c90 2023-07-06T20:14:17.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.040+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8003c20 con 0x7f83b80b5c90 2023-07-06T20:14:17.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.040+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b80cb5d0 con 0x7f83b80b5c90 2023-07-06T20:14:17.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.041+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b80cb7c0 con 0x7f83b80b5c90 2023-07-06T20:14:17.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.042+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83a8025480 con 0x7f83b80b5c90 2023-07-06T20:14:17.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.043+0000 7f839ffff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f83a406ace0 0x7f83a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.043+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83a8020070 con 0x7f83b80b5c90 2023-07-06T20:14:17.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.044+0000 7f83b6ffd700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f83a406ace0 0x7f83a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.044+0000 7f83b6ffd700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f83a406ace0 0x7f83a406d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac007990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.048+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83a8055b10 con 0x7f83b80b5c90 2023-07-06T20:14:17.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.257+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83b80008d0 con 0x7f83b80b5c90 2023-07-06T20:14:17.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.257+0000 7f839ffff700 1 -- 172.21.15.27:0/820892330 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f83a8015020 con 0x7f83b80b5c90 2023-07-06T20:14:17.260 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:17.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.259+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f83a406ace0 msgr2=0x7f83a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.259+0000 7f83be1ff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f83a406ace0 0x7f83a406d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac007990 comp rx=0 tx=0).stop 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 msgr2=0x7f83b815ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 0x7f83b815ad80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83a8014040 tx=0x7f83a8004750 comp rx=0 tx=0).stop 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 shutdown_connections 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f83a406ace0 0x7f83a406d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83b815fca0 0x7f83b814a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83b80b5c90 0x7f83b815ad80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 --2- 172.21.15.27:0/820892330 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f83b80b6110 0x7f83b80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 >> 172.21.15.27:0/820892330 conn(0x7f83b806c150 msgr2=0x7f83b80b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:17.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 shutdown_connections 2023-07-06T20:14:17.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.260+0000 7f83be1ff700 1 -- 172.21.15.27:0/820892330 wait complete. 2023-07-06T20:14:17.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.485+0000 7fb4493ff700 1 Processor -- start 2023-07-06T20:14:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.485+0000 7fb4493ff700 1 -- start start 2023-07-06T20:14:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb4493ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4440b5c90 0x7fb4440d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb4493ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb4440b6110 0x7fb4440d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb4493ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440cf780 con 0x7fb44415fca0 2023-07-06T20:14:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb4493ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440cd490 con 0x7fb4440b5c90 2023-07-06T20:14:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb4493ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440cd770 con 0x7fb4440b6110 2023-07-06T20:14:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb43bfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4440b5c90 0x7fb4440d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb43bfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4440b5c90 0x7fb4440d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44994/0 (socket says 172.21.15.27:44994) 2023-07-06T20:14:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb43bfff700 1 -- 172.21.15.27:0/836030727 learned_addr learned my addr 172.21.15.27:0/836030727 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.486+0000 7fb4437fe700 1 --2- 172.21.15.27:0/836030727 >> v2:172.21.15.153:3300/0 conn(0x7fb4440b6110 0x7fb4440d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb43bfff700 1 -- 172.21.15.27:0/836030727 >> v2:172.21.15.153:3300/0 conn(0x7fb4440b6110 msgr2=0x7fb4440d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:17.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb43bfff700 1 --2- 172.21.15.27:0/836030727 >> v2:172.21.15.153:3300/0 conn(0x7fb4440b6110 0x7fb4440d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb43bfff700 1 -- 172.21.15.27:0/836030727 >> v1:172.21.15.153:6789/0 conn(0x7fb44415fca0 legacy=0x7fb4440d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb43bfff700 1 -- 172.21.15.27:0/836030727 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440cda50 con 0x7fb4440b5c90 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb4437fe700 1 --2- 172.21.15.27:0/836030727 >> v2:172.21.15.153:3300/0 conn(0x7fb4440b6110 0x7fb4440d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb43bfff700 1 --2- 172.21.15.27:0/836030727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440b5c90 0x7fb4440d2d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4440cd410 tx=0x7fb42c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f7d6a8ea7edfe4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb4427fc700 1 -- 172.21.15.27:0/836030727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00b870 con 0x7fb4440b5c90 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb4427fc700 1 -- 172.21.15.27:0/836030727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c00b9d0 con 0x7fb4440b5c90 2023-07-06T20:14:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.487+0000 7fb4427fc700 1 -- 172.21.15.27:0/836030727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00bd60 con 0x7fb4440b5c90 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 -- 172.21.15.27:0/836030727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440b5c90 msgr2=0x7fb4440d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 --2- 172.21.15.27:0/836030727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440b5c90 0x7fb4440d2d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4440cd410 tx=0x7fb42c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 -- 172.21.15.27:0/836030727 shutdown_connections 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 --2- 172.21.15.27:0/836030727 >> v2:172.21.15.153:3300/0 conn(0x7fb4440b6110 0x7fb4440d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 --2- 172.21.15.27:0/836030727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440b5c90 0x7fb4440d2d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 -- 172.21.15.27:0/836030727 >> 172.21.15.27:0/836030727 conn(0x7fb44406c150 msgr2=0x7fb44406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 -- 172.21.15.27:0/836030727 shutdown_connections 2023-07-06T20:14:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.488+0000 7fb4493ff700 1 -- 172.21.15.27:0/836030727 wait complete. 2023-07-06T20:14:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.490+0000 7fb4493ff700 1 Processor -- start 2023-07-06T20:14:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.490+0000 7fb4493ff700 1 -- start start 2023-07-06T20:14:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4493ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4440b5c90 0x7fb4441542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4493ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4440b6110 0x7fb4440cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4493ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 0x7fb4440cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4493ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440c6b30 con 0x7fb44415fca0 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4493ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444154bd0 con 0x7fb4440b6110 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4493ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb444154eb0 con 0x7fb4440b5c90 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 0x7fb4440cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb442ffd700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4440b5c90 0x7fb4441542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb43bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4440b6110 0x7fb4440cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 0x7fb4440cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45008/0 (socket says 172.21.15.27:45008) 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.491+0000 7fb4437fe700 1 -- 172.21.15.27:0/2853087123 learned_addr learned my addr 172.21.15.27:0/2853087123 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4437fe700 1 -- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4440b5c90 msgr2=0x7fb4441542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4437fe700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4440b5c90 0x7fb4441542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4437fe700 1 -- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4440b6110 msgr2=0x7fb4440cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4437fe700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4440b6110 0x7fb4440cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4437fe700 1 -- 172.21.15.27:0/2853087123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009550 con 0x7fb44415fca0 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4437fe700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 0x7fb4440cc510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb43c007ef0 tx=0x7fb43c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00dd80 con 0x7fb44415fca0 2023-07-06T20:14:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c00f040 con 0x7fb44415fca0 2023-07-06T20:14:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb444155190 con 0x7fb44415fca0 2023-07-06T20:14:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c003c60 con 0x7fb44415fca0 2023-07-06T20:14:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.492+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb444155750 con 0x7fb44415fca0 2023-07-06T20:14:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.493+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb444155940 con 0x7fb44415fca0 2023-07-06T20:14:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.494+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb43c004030 con 0x7fb44415fca0 2023-07-06T20:14:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.496+0000 7fb43b7fe700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb42406ace0 0x7fb42406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.496+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb43c089540 con 0x7fb44415fca0 2023-07-06T20:14:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.496+0000 7fb442ffd700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb42406ace0 0x7fb42406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.497+0000 7fb442ffd700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb42406ace0 0x7fb42406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb434007fd0 tx=0x7fb434009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.500+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb43c0545e0 con 0x7fb44415fca0 2023-07-06T20:14:17.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.705+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4440008d0 con 0x7fb44415fca0 2023-07-06T20:14:17.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.706+0000 7fb43b7fe700 1 -- 172.21.15.27:0/2853087123 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb43c057c00 con 0x7fb44415fca0 2023-07-06T20:14:17.708 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:17.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb42406ace0 msgr2=0x7fb42406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb42406ace0 0x7fb42406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb434007fd0 tx=0x7fb434009040 comp rx=0 tx=0).stop 2023-07-06T20:14:17.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 msgr2=0x7fb4440cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 0x7fb4440cc510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb43c007ef0 tx=0x7fb43c00d3b0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 shutdown_connections 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb42406ace0 0x7fb42406d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44415fca0 0x7fb4440cc510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4440b5c90 0x7fb4441542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 --2- 172.21.15.27:0/2853087123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4440b6110 0x7fb4440cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 >> 172.21.15.27:0/2853087123 conn(0x7fb44406c150 msgr2=0x7fb4440d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 shutdown_connections 2023-07-06T20:14:17.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.708+0000 7fb4493ff700 1 -- 172.21.15.27:0/2853087123 wait complete. 2023-07-06T20:14:17.728 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.934+0000 7efbfcfff700 1 Processor -- start 2023-07-06T20:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.934+0000 7efbfcfff700 1 -- start start 2023-07-06T20:14:17.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbfcfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf815fca0 0x7efbf80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbfcfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efbf80b5c90 0x7efbf80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbfcfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efbf80b6100 0x7efbf80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbfcfff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80cf780 con 0x7efbf80b6100 2023-07-06T20:14:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbfcfff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80cd490 con 0x7efbf80b5c90 2023-07-06T20:14:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbfcfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80cd770 con 0x7efbf815fca0 2023-07-06T20:14:17.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbf6d9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efbf80b6100 0x7efbf80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbf6d9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efbf80b6100 0x7efbf80d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45022/0 (socket says 172.21.15.27:45022) 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbf6d9d700 1 -- 172.21.15.27:0/3365989938 learned_addr learned my addr 172.21.15.27:0/3365989938 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbf659c700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf815fca0 0x7efbf80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.935+0000 7efbeffff700 1 --2- 172.21.15.27:0/3365989938 >> v2:172.21.15.153:3300/0 conn(0x7efbf80b5c90 0x7efbf80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf6d9d700 1 -- 172.21.15.27:0/3365989938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf815fca0 msgr2=0x7efbf80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf6d9d700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf815fca0 0x7efbf80d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf6d9d700 1 -- 172.21.15.27:0/3365989938 >> v2:172.21.15.153:3300/0 conn(0x7efbf80b5c90 msgr2=0x7efbf80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf6d9d700 1 --2- 172.21.15.27:0/3365989938 >> v2:172.21.15.153:3300/0 conn(0x7efbf80b5c90 0x7efbf80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf6d9d700 1 -- 172.21.15.27:0/3365989938 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80cda50 con 0x7efbf80b6100 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbeffff700 1 --2- 172.21.15.27:0/3365989938 >> v2:172.21.15.153:3300/0 conn(0x7efbf80b5c90 0x7efbf80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf659c700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf815fca0 0x7efbf80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf6d9d700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf80b6100 0x7efbf80d3280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efbf806cda0 tx=0x7efbf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59a9c3dfcaaaed server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf5d9b700 1 -- 172.21.15.27:0/3365989938 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000e070 con 0x7efbf80b6100 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf5d9b700 1 -- 172.21.15.27:0/3365989938 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf000b9c0 con 0x7efbf80b6100 2023-07-06T20:14:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.936+0000 7efbf5d9b700 1 -- 172.21.15.27:0/3365989938 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000bd70 con 0x7efbf80b6100 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 -- 172.21.15.27:0/3365989938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf80b6100 msgr2=0x7efbf80d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf80b6100 0x7efbf80d3280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efbf806cda0 tx=0x7efbf000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 -- 172.21.15.27:0/3365989938 shutdown_connections 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf80b6100 0x7efbf80d3280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 --2- 172.21.15.27:0/3365989938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf815fca0 0x7efbf80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 --2- 172.21.15.27:0/3365989938 >> v2:172.21.15.153:3300/0 conn(0x7efbf80b5c90 0x7efbf80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 -- 172.21.15.27:0/3365989938 >> 172.21.15.27:0/3365989938 conn(0x7efbf806c150 msgr2=0x7efbf806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 -- 172.21.15.27:0/3365989938 shutdown_connections 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.937+0000 7efbfcfff700 1 -- 172.21.15.27:0/3365989938 wait complete. 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.939+0000 7efbfcfff700 1 Processor -- start 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.939+0000 7efbfcfff700 1 -- start start 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbfcfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf80b5c90 0x7efbf80c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbfcfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf80b6100 0x7efbf815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbfcfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 0x7efbf8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbfcfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80c90f0 con 0x7efbf815fca0 2023-07-06T20:14:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbfcfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8149240 con 0x7efbf80b5c90 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbfcfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8149520 con 0x7efbf80b6100 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 0x7efbf8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 0x7efbf8158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45038/0 (socket says 172.21.15.27:45038) 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 -- 172.21.15.27:0/2783292084 learned_addr learned my addr 172.21.15.27:0/2783292084 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbeffff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf80b6100 0x7efbf815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf659c700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf80b5c90 0x7efbf80c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 -- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf80b6100 msgr2=0x7efbf815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf80b6100 0x7efbf815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 -- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf80b5c90 msgr2=0x7efbf80c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf80b5c90 0x7efbf80c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.940+0000 7efbf6d9d700 1 -- 172.21.15.27:0/2783292084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf0009b30 con 0x7efbf815fca0 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.941+0000 7efbf6d9d700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 0x7efbf8158ee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efbf0012000 tx=0x7efbf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.941+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0004250 con 0x7efbf815fca0 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.941+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf00043d0 con 0x7efbf815fca0 2023-07-06T20:14:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.941+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8149800 con 0x7efbf815fca0 2023-07-06T20:14:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.941+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0004780 con 0x7efbf815fca0 2023-07-06T20:14:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.941+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf8149dc0 con 0x7efbf815fca0 2023-07-06T20:14:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.942+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf80008d0 con 0x7efbf815fca0 2023-07-06T20:14:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.943+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efbf001b430 con 0x7efbf815fca0 2023-07-06T20:14:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.944+0000 7efbeeffd700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efbd806ace0 0x7efbd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.944+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efbf008a640 con 0x7efbf815fca0 2023-07-06T20:14:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.944+0000 7efbf659c700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efbd806ace0 0x7efbd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.945+0000 7efbf659c700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efbd806ace0 0x7efbd806d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009930 tx=0x7efbe8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:17.948+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efbf0055760 con 0x7efbf815fca0 2023-07-06T20:14:18.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.157+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efbf80b7080 con 0x7efbf815fca0 2023-07-06T20:14:18.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.157+0000 7efbeeffd700 1 -- 172.21.15.27:0/2783292084 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7efbf0058d80 con 0x7efbf815fca0 2023-07-06T20:14:18.159 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efbd806ace0 msgr2=0x7efbd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efbd806ace0 0x7efbd806d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009930 tx=0x7efbe8008040 comp rx=0 tx=0).stop 2023-07-06T20:14:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 msgr2=0x7efbf8158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 0x7efbf8158ee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efbf0012000 tx=0x7efbf0004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 shutdown_connections 2023-07-06T20:14:18.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efbd806ace0 0x7efbd806d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efbf815fca0 0x7efbf8158ee0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf80b5c90 0x7efbf80c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 --2- 172.21.15.27:0/2783292084 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efbf80b6100 0x7efbf815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 >> 172.21.15.27:0/2783292084 conn(0x7efbf806c150 msgr2=0x7efbf80d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:18.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 shutdown_connections 2023-07-06T20:14:18.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.160+0000 7efbfcfff700 1 -- 172.21.15.27:0/2783292084 wait complete. 2023-07-06T20:14:18.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:14:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.381+0000 7f5eabfff700 1 Processor -- start 2023-07-06T20:14:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.381+0000 7f5eabfff700 1 -- start start 2023-07-06T20:14:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.382+0000 7f5eabfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5ea415fca0 0x7f5ea40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.382+0000 7f5eabfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.382+0000 7f5eabfff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40cf780 con 0x7f5ea40b60e0 2023-07-06T20:14:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.382+0000 7f5eabfff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea40cd490 con 0x7f5ea415fca0 2023-07-06T20:14:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.382+0000 7f5eabfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40cd770 con 0x7f5ea40b5c90 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea959a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5ea415fca0 0x7f5ea40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5ea415fca0 0x7f5ea40d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55346/0 (socket says 172.21.15.27:55346) 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea959a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:48008/0 (socket says 172.21.15.27:48008) 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 -- 172.21.15.27:0/3964948214 learned_addr learned my addr 172.21.15.27:0/3964948214 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5eaa59c700 1 --1- 172.21.15.27:0/3964948214 >> v1:172.21.15.153:6789/0 conn(0x7f5ea40b60e0 0x7f5ea40cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48804/0 (socket says 172.21.15.27:48804) 2023-07-06T20:14:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea8d99700 1 -- 172.21.15.27:0/3964948214 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718963144 0 0) 0x7f5ea40cf780 con 0x7f5ea40b60e0 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea8d99700 1 -- 172.21.15.27:0/3964948214 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e900035b0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 -- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 msgr2=0x7f5ea40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 --2- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 -- 172.21.15.27:0/3964948214 >> v1:172.21.15.153:6789/0 conn(0x7f5ea40b60e0 legacy=0x7f5ea40cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea9d9b700 1 -- 172.21.15.27:0/3964948214 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40cda50 con 0x7f5ea415fca0 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.383+0000 7f5ea959a700 1 --2- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.384+0000 7f5ea9d9b700 1 --2- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea40d1420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00098f0 tx=0x7f5ea0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb73da75584b0a1e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.384+0000 7f5ea8d99700 1 -- 172.21.15.27:0/3964948214 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0011830 con 0x7f5ea415fca0 2023-07-06T20:14:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.384+0000 7f5ea8d99700 1 -- 172.21.15.27:0/3964948214 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea0011990 con 0x7f5ea415fca0 2023-07-06T20:14:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.384+0000 7f5ea8d99700 1 -- 172.21.15.27:0/3964948214 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0011d20 con 0x7f5ea415fca0 2023-07-06T20:14:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.384+0000 7f5eabfff700 1 -- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 msgr2=0x7f5ea40d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.384+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea40d1420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00098f0 tx=0x7f5ea0009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.385+0000 7f5eabfff700 1 -- 172.21.15.27:0/3964948214 shutdown_connections 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.385+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea40d1420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.385+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3964948214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.385+0000 7f5eabfff700 1 -- 172.21.15.27:0/3964948214 >> 172.21.15.27:0/3964948214 conn(0x7f5ea406c150 msgr2=0x7f5ea406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.385+0000 7f5eabfff700 1 -- 172.21.15.27:0/3964948214 shutdown_connections 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.385+0000 7f5eabfff700 1 -- 172.21.15.27:0/3964948214 wait complete. 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.386+0000 7f5eabfff700 1 Processor -- start 2023-07-06T20:14:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.386+0000 7f5eabfff700 1 -- start start 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.387+0000 7f5eabfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.387+0000 7f5eabfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 0x7f5ea41ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.387+0000 7f5eabfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea41f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.387+0000 7f5eabfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f5ab0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.387+0000 7f5eabfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f5d50 con 0x7f5ea415fca0 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.387+0000 7f5eabfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f6030 con 0x7f5ea40b5c90 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 0x7f5ea41ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 0x7f5ea41ef5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45052/0 (socket says 172.21.15.27:45052) 2023-07-06T20:14:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 -- 172.21.15.27:0/3987431804 learned_addr learned my addr 172.21.15.27:0/3987431804 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea9d9b700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5eaa59c700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea41f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 -- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 msgr2=0x7f5ea40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 -- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 msgr2=0x7f5ea41f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea41f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 -- 172.21.15.27:0/3987431804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940094a0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.388+0000 7f5ea959a700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 0x7f5ea41ef5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400ebe0 tx=0x7f5e94009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.389+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400c910 con 0x7f5ea40b60e0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.389+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9400ca70 con 0x7f5ea40b60e0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.389+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400cde0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.389+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea0009550 con 0x7f5ea40b60e0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.389+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea41f1f80 con 0x7f5ea40b60e0 2023-07-06T20:14:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.390+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea41f2340 con 0x7f5ea40b60e0 2023-07-06T20:14:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.391+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e94013470 con 0x7f5ea40b60e0 2023-07-06T20:14:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.392+0000 7f5e9a7fc700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e8806ace0 0x7f5e8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.392+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e940897c0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.392+0000 7f5ea9d9b700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e8806ace0 0x7f5e8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.393+0000 7f5ea9d9b700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e8806ace0 0x7f5e8806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0014040 tx=0x7f5ea0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.396+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e940547d0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.647+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea40008d0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.647+0000 7f5e9a7fc700 1 -- 172.21.15.27:0/3987431804 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f5e94057df0 con 0x7f5ea40b60e0 2023-07-06T20:14:18.649 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:18.650 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:14:18.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.649+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e8806ace0 msgr2=0x7f5e8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.649+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e8806ace0 0x7f5e8806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0014040 tx=0x7f5ea0008030 comp rx=0 tx=0).stop 2023-07-06T20:14:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 msgr2=0x7f5ea41ef5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 0x7f5ea41ef5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400ebe0 tx=0x7f5e94009ec0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 shutdown_connections 2023-07-06T20:14:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e8806ace0 0x7f5e8806d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ea415fca0 0x7f5ea41f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ea40b5c90 0x7f5ea40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 --2- 172.21.15.27:0/3987431804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea40b60e0 0x7f5ea41ef5a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 >> 172.21.15.27:0/3987431804 conn(0x7f5ea406c150 msgr2=0x7f5ea40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 shutdown_connections 2023-07-06T20:14:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.650+0000 7f5eabfff700 1 -- 172.21.15.27:0/3987431804 wait complete. 2023-07-06T20:14:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:14:18.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:18.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.870+0000 7f3ec49ff700 1 Processor -- start 2023-07-06T20:14:18.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.871+0000 7f3ec49ff700 1 -- start start 2023-07-06T20:14:18.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ec49ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec015fca0 0x7f3ec00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ec49ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3ec00b6100 0x7f3ec00cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ec49ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec00cf780 con 0x7f3ec00b5c90 2023-07-06T20:14:18.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ec49ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec00cd490 con 0x7f3ec00b6100 2023-07-06T20:14:18.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ec49ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00cd770 con 0x7f3ec015fca0 2023-07-06T20:14:18.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ebe59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec015fca0 0x7f3ec00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3ebed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3ec00b6100 0x7f3ec00cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3eb7fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3ec00b5c90 0x7f3ec00d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48806/0 (socket says 172.21.15.27:48806) 2023-07-06T20:14:18.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.872+0000 7f3eb7fff700 1 -- 172.21.15.27:0/809877428 learned_addr learned my addr 172.21.15.27:0/809877428 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebed9d700 1 -- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec015fca0 msgr2=0x7f3ec00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebed9d700 1 --2- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec015fca0 0x7f3ec00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebed9d700 1 -- 172.21.15.27:0/809877428 >> v1:172.21.15.153:6789/0 conn(0x7f3ec00b5c90 legacy=0x7f3ec00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebed9d700 1 -- 172.21.15.27:0/809877428 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec00cda50 con 0x7f3ec00b6100 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebdd9b700 1 -- 172.21.15.27:0/809877428 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609465290 0 0) 0x7f3ec00cf780 con 0x7f3ec00b5c90 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebed9d700 1 --2- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec00b6100 0x7f3ec00cfc10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009d90 tx=0x7f3eb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e77bdff54d87453 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:18.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebdd9b700 1 -- 172.21.15.27:0/809877428 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb800e070 con 0x7f3ec00b6100 2023-07-06T20:14:18.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebdd9b700 1 -- 172.21.15.27:0/809877428 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb800bb20 con 0x7f3ec00b6100 2023-07-06T20:14:18.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.873+0000 7f3ebdd9b700 1 -- 172.21.15.27:0/809877428 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8012430 con 0x7f3ec00b6100 2023-07-06T20:14:18.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 -- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec00b6100 msgr2=0x7f3ec00cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec00b6100 0x7f3ec00cfc10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009d90 tx=0x7f3eb800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 -- 172.21.15.27:0/809877428 shutdown_connections 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec015fca0 0x7f3ec00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/809877428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec00b6100 0x7f3ec00cfc10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 -- 172.21.15.27:0/809877428 >> 172.21.15.27:0/809877428 conn(0x7f3ec006c150 msgr2=0x7f3ec006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 -- 172.21.15.27:0/809877428 shutdown_connections 2023-07-06T20:14:18.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.874+0000 7f3ec49ff700 1 -- 172.21.15.27:0/809877428 wait complete. 2023-07-06T20:14:18.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.876+0000 7f3ec49ff700 1 Processor -- start 2023-07-06T20:14:18.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.876+0000 7f3ec49ff700 1 -- start start 2023-07-06T20:14:18.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ec49ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec00b5c90 0x7f3ec00cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ec49ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 0x7f3ec0158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ec49ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec015fca0 0x7f3ec01542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ec49ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec015c400 con 0x7f3ec00b6100 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ec49ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00ca4b0 con 0x7f3ec015fca0 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ec49ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec00ca790 con 0x7f3ec00b5c90 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3eb7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 0x7f3ec0158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3eb7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 0x7f3ec0158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45058/0 (socket says 172.21.15.27:45058) 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3eb7fff700 1 -- 172.21.15.27:0/213794105 learned_addr learned my addr 172.21.15.27:0/213794105 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ebe59c700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec00b5c90 0x7f3ec00cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.877+0000 7f3ebed9d700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec015fca0 0x7f3ec01542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb7fff700 1 -- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec00b5c90 msgr2=0x7f3ec00cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb7fff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec00b5c90 0x7f3ec00cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb7fff700 1 -- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec015fca0 msgr2=0x7f3ec01542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb7fff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec015fca0 0x7f3ec01542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb7fff700 1 -- 172.21.15.27:0/213794105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb8009a70 con 0x7f3ec00b6100 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb7fff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 0x7f3ec0158e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80011e0 tx=0x7f3ea800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8004c80 con 0x7f3ec00b6100 2023-07-06T20:14:18.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8004e00 con 0x7f3ec00b6100 2023-07-06T20:14:18.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8007310 con 0x7f3ec00b6100 2023-07-06T20:14:18.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec00caa70 con 0x7f3ec00b6100 2023-07-06T20:14:18.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.878+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec00b86a0 con 0x7f3ec00b6100 2023-07-06T20:14:18.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.879+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec015c570 con 0x7f3ec00b6100 2023-07-06T20:14:18.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.880+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ea8007470 con 0x7f3ec00b6100 2023-07-06T20:14:18.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.881+0000 7f3eb6ffd700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ea006ace0 0x7f3ea006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:18.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.881+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ea8089130 con 0x7f3ec00b6100 2023-07-06T20:14:18.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.882+0000 7f3ebe59c700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ea006ace0 0x7f3ea006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:18.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.882+0000 7f3ebe59c700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ea006ace0 0x7f3ea006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009b40 tx=0x7f3eb0009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:18.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:18.885+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ea80541d0 con 0x7f3ec00b6100 2023-07-06T20:14:19.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.100+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ec00008d0 con 0x7f3ec00b6100 2023-07-06T20:14:19.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.101+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/213794105 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f3ea80577f0 con 0x7f3ec00b6100 2023-07-06T20:14:19.103 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:19.103 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ea006ace0 msgr2=0x7f3ea006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ea006ace0 0x7f3ea006d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009b40 tx=0x7f3eb0009210 comp rx=0 tx=0).stop 2023-07-06T20:14:19.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 msgr2=0x7f3ec0158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 0x7f3ec0158e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80011e0 tx=0x7f3ea800ca30 comp rx=0 tx=0).stop 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 shutdown_connections 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3ea006ace0 0x7f3ea006d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec015fca0 0x7f3ec01542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3ec00b5c90 0x7f3ec00cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 --2- 172.21.15.27:0/213794105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec00b6100 0x7f3ec0158e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.103+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 >> 172.21.15.27:0/213794105 conn(0x7f3ec006c150 msgr2=0x7f3ec00d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.104+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 shutdown_connections 2023-07-06T20:14:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.104+0000 7f3ec49ff700 1 -- 172.21.15.27:0/213794105 wait complete. 2023-07-06T20:14:19.123 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:19.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.327+0000 7fa848bff700 1 Processor -- start 2023-07-06T20:14:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.328+0000 7fa848bff700 1 -- start start 2023-07-06T20:14:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa848bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 0x7fa8440d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa848bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa848bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8440cf780 con 0x7fa8440b60e0 2023-07-06T20:14:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa848bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8440cd490 con 0x7fa84415fca0 2023-07-06T20:14:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa848bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440cd770 con 0x7fa8440b5c90 2023-07-06T20:14:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa83bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa84259c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 0x7fa8440d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa83bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:48054/0 (socket says 172.21.15.27:48054) 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa84259c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 0x7fa8440d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:55364/0 (socket says 172.21.15.27:55364) 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa84259c700 1 -- 172.21.15.27:0/2687388953 learned_addr learned my addr 172.21.15.27:0/2687388953 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.329+0000 7fa842d9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa8440b60e0 0x7fa8440cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48820/0 (socket says 172.21.15.27:48820) 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa841d9b700 1 -- 172.21.15.27:0/2687388953 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191492143 0 0) 0x7fa8440cf780 con 0x7fa8440b60e0 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa83bfff700 1 -- 172.21.15.27:0/2687388953 >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 msgr2=0x7fa8440d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa83bfff700 1 --2- 172.21.15.27:0/2687388953 >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 0x7fa8440d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa83bfff700 1 -- 172.21.15.27:0/2687388953 >> v1:172.21.15.153:6789/0 conn(0x7fa8440b60e0 legacy=0x7fa8440cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa83bfff700 1 -- 172.21.15.27:0/2687388953 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8440cda50 con 0x7fa8440b5c90 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa84259c700 1 --2- 172.21.15.27:0/2687388953 >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 0x7fa8440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa83bfff700 1 --2- 172.21.15.27:0/2687388953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440d2d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009840 tx=0x7fa82c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470eaa668b71049d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa841d9b700 1 -- 172.21.15.27:0/2687388953 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c011830 con 0x7fa8440b5c90 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa841d9b700 1 -- 172.21.15.27:0/2687388953 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c011990 con 0x7fa8440b5c90 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.330+0000 7fa841d9b700 1 -- 172.21.15.27:0/2687388953 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c011d20 con 0x7fa8440b5c90 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 -- 172.21.15.27:0/2687388953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 msgr2=0x7fa8440d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 --2- 172.21.15.27:0/2687388953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440d2d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009840 tx=0x7fa82c009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 -- 172.21.15.27:0/2687388953 shutdown_connections 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 --2- 172.21.15.27:0/2687388953 >> v2:172.21.15.153:3300/0 conn(0x7fa84415fca0 0x7fa8440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 --2- 172.21.15.27:0/2687388953 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440d2d40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 -- 172.21.15.27:0/2687388953 >> 172.21.15.27:0/2687388953 conn(0x7fa84406c150 msgr2=0x7fa84406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 -- 172.21.15.27:0/2687388953 shutdown_connections 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.331+0000 7fa848bff700 1 -- 172.21.15.27:0/2687388953 wait complete. 2023-07-06T20:14:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.333+0000 7fa848bff700 1 Processor -- start 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.333+0000 7fa848bff700 1 -- start start 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa848bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa848bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8440b60e0 0x7fa8440a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa848bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 0x7fa8440a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa848bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440b5260 con 0x7fa84415fca0 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa848bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844149ba0 con 0x7fa8440b60e0 2023-07-06T20:14:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa848bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa844149e80 con 0x7fa8440b5c90 2023-07-06T20:14:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa842d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 0x7fa8440a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa842d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 0x7fa8440a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45070/0 (socket says 172.21.15.27:45070) 2023-07-06T20:14:19.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa842d9d700 1 -- 172.21.15.27:0/394551512 learned_addr learned my addr 172.21.15.27:0/394551512 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa84259c700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.334+0000 7fa83bfff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8440b60e0 0x7fa8440a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa842d9d700 1 -- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 msgr2=0x7fa8440c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa842d9d700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa842d9d700 1 -- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8440b60e0 msgr2=0x7fa8440a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa842d9d700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8440b60e0 0x7fa8440a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa842d9d700 1 -- 172.21.15.27:0/394551512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009550 con 0x7fa84415fca0 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa842d9d700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 0x7fa8440a9cf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00cc10 tx=0x7fa83c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00abb0 con 0x7fa84415fca0 2023-07-06T20:14:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c00ad10 con 0x7fa84415fca0 2023-07-06T20:14:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007310 con 0x7fa84415fca0 2023-07-06T20:14:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0094a0 con 0x7fa84415fca0 2023-07-06T20:14:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.335+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84414a440 con 0x7fa84415fca0 2023-07-06T20:14:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.336+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8440008d0 con 0x7fa84415fca0 2023-07-06T20:14:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.337+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa83c007470 con 0x7fa84415fca0 2023-07-06T20:14:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.338+0000 7fa83affd700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa82406ace0 0x7fa82406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.338+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa83c089130 con 0x7fa84415fca0 2023-07-06T20:14:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.339+0000 7fa84259c700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa82406ace0 0x7fa82406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.340+0000 7fa84259c700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa82406ace0 0x7fa82406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa834009c50 tx=0x7fa834008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.343+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa83c054140 con 0x7fa84415fca0 2023-07-06T20:14:19.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.551+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa84415b150 con 0x7fa84415fca0 2023-07-06T20:14:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.551+0000 7fa83affd700 1 -- 172.21.15.27:0/394551512 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa83c057760 con 0x7fa84415fca0 2023-07-06T20:14:19.553 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa82406ace0 msgr2=0x7fa82406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa82406ace0 0x7fa82406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa834009c50 tx=0x7fa834008040 comp rx=0 tx=0).stop 2023-07-06T20:14:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 msgr2=0x7fa8440a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 0x7fa8440a9cf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00cc10 tx=0x7fa83c00a060 comp rx=0 tx=0).stop 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 shutdown_connections 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa82406ace0 0x7fa82406d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa84415fca0 0x7fa8440a9cf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440b5c90 0x7fa8440c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 --2- 172.21.15.27:0/394551512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8440b60e0 0x7fa8440a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 >> 172.21.15.27:0/394551512 conn(0x7fa84406c150 msgr2=0x7fa8440b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 shutdown_connections 2023-07-06T20:14:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.554+0000 7fa848bff700 1 -- 172.21.15.27:0/394551512 wait complete. 2023-07-06T20:14:19.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:19.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.778+0000 7f725b9ff700 1 Processor -- start 2023-07-06T20:14:19.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.778+0000 7f725b9ff700 1 -- start start 2023-07-06T20:14:19.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f725b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725415fca0 0x7f72540d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f725b9ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f72540b6100 0x7f72540cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f725b9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540cf780 con 0x7f72540b5c90 2023-07-06T20:14:19.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f725b9ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540cd490 con 0x7f72540b6100 2023-07-06T20:14:19.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f725b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cd770 con 0x7f725415fca0 2023-07-06T20:14:19.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f7259f9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f72540b6100 0x7f72540cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f7259f9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f72540b6100 0x7f72540cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45080/0 (socket says 172.21.15.27:45080) 2023-07-06T20:14:19.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f7259f9c700 1 -- 172.21.15.27:0/1044111539 learned_addr learned my addr 172.21.15.27:0/1044111539 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:19.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.779+0000 7f725979b700 1 --2- 172.21.15.27:0/1044111539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725415fca0 0x7f72540d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f7259f9c700 1 -- 172.21.15.27:0/1044111539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725415fca0 msgr2=0x7f72540d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:19.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f7259f9c700 1 --2- 172.21.15.27:0/1044111539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725415fca0 0x7f72540d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f7259f9c700 1 -- 172.21.15.27:0/1044111539 >> v1:172.21.15.153:6789/0 conn(0x7f72540b5c90 legacy=0x7f72540d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f7259f9c700 1 -- 172.21.15.27:0/1044111539 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540cda50 con 0x7f72540b6100 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f7259f9c700 1 --2- 172.21.15.27:0/1044111539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b6100 0x7f72540cfc10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f724c009e50 tx=0x7f724c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36142af0f3ce776f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f724bfff700 1 -- 172.21.15.27:0/1044111539 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00e070 con 0x7f72540b6100 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f724bfff700 1 -- 172.21.15.27:0/1044111539 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00b910 con 0x7f72540b6100 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.780+0000 7f724bfff700 1 -- 172.21.15.27:0/1044111539 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00bcc0 con 0x7f72540b6100 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 -- 172.21.15.27:0/1044111539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b6100 msgr2=0x7f72540cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 --2- 172.21.15.27:0/1044111539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b6100 0x7f72540cfc10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f724c009e50 tx=0x7f724c00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 -- 172.21.15.27:0/1044111539 shutdown_connections 2023-07-06T20:14:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 --2- 172.21.15.27:0/1044111539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b6100 0x7f72540cfc10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 --2- 172.21.15.27:0/1044111539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725415fca0 0x7f72540d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 -- 172.21.15.27:0/1044111539 >> 172.21.15.27:0/1044111539 conn(0x7f725406c150 msgr2=0x7f725406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 -- 172.21.15.27:0/1044111539 shutdown_connections 2023-07-06T20:14:19.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.781+0000 7f725b9ff700 1 -- 172.21.15.27:0/1044111539 wait complete. 2023-07-06T20:14:19.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.783+0000 7f725b9ff700 1 Processor -- start 2023-07-06T20:14:19.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.783+0000 7f725b9ff700 1 -- start start 2023-07-06T20:14:19.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725b9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 0x7f72540b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72540b6100 0x7f72540b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725b9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f725415fca0 0x7f725415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725b9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72541f1f90 con 0x7f72540b5c90 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725b9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72541f2230 con 0x7f725415fca0 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72541f2510 con 0x7f72540b6100 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725979b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 0x7f72540b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725979b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 0x7f72540b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45086/0 (socket says 172.21.15.27:45086) 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f725979b700 1 -- 172.21.15.27:0/2791761110 learned_addr learned my addr 172.21.15.27:0/2791761110 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f7258f9a700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72540b6100 0x7f72540b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.784+0000 7f7259f9c700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f725415fca0 0x7f725415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725979b700 1 -- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72540b6100 msgr2=0x7f72540b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725979b700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72540b6100 0x7f72540b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725979b700 1 -- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f725415fca0 msgr2=0x7f725415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725979b700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f725415fca0 0x7f725415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725979b700 1 -- 172.21.15.27:0/2791761110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c009b30 con 0x7f72540b5c90 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f7258f9a700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72540b6100 0x7f72540b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725979b700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 0x7f72540b2090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f725000e870 tx=0x7f725000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250009590 con 0x7f72540b5c90 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725000ee70 con 0x7f72540b5c90 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72541f27f0 con 0x7f72540b5c90 2023-07-06T20:14:19.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725000ca50 con 0x7f72540b5c90 2023-07-06T20:14:19.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.785+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72540b25d0 con 0x7f72540b5c90 2023-07-06T20:14:19.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.786+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72540008d0 con 0x7f72540b5c90 2023-07-06T20:14:19.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.787+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f725000cbb0 con 0x7f72540b5c90 2023-07-06T20:14:19.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.789+0000 7f7249ffb700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723806ace0 0x7f723806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:19.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.789+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72500894f0 con 0x7f72540b5c90 2023-07-06T20:14:19.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.789+0000 7f7258f9a700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723806ace0 0x7f723806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:19.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.790+0000 7f7258f9a700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723806ace0 0x7f723806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7244000f60 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:19.793+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7250057f90 con 0x7f72540b5c90 2023-07-06T20:14:20.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.001+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72540b2c40 con 0x7f72540b5c90 2023-07-06T20:14:20.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.002+0000 7f7249ffb700 1 -- 172.21.15.27:0/2791761110 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f725000ce60 con 0x7f72540b5c90 2023-07-06T20:14:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:20.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723806ace0 msgr2=0x7f723806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723806ace0 0x7f723806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7244000f60 tx=0x7f7244009040 comp rx=0 tx=0).stop 2023-07-06T20:14:20.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 msgr2=0x7f72540b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 0x7f72540b2090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f725000e870 tx=0x7f725000ec30 comp rx=0 tx=0).stop 2023-07-06T20:14:20.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 shutdown_connections 2023-07-06T20:14:20.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723806ace0 0x7f723806d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f725415fca0 0x7f725415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72540b5c90 0x7f72540b2090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 --2- 172.21.15.27:0/2791761110 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f72540b6100 0x7f72540b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 >> 172.21.15.27:0/2791761110 conn(0x7f725406c150 msgr2=0x7f72540b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:20.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.004+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 shutdown_connections 2023-07-06T20:14:20.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.005+0000 7f725b9ff700 1 -- 172.21.15.27:0/2791761110 wait complete. 2023-07-06T20:14:20.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.230+0000 7f293bbff700 1 Processor -- start 2023-07-06T20:14:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.230+0000 7f293bbff700 1 -- start start 2023-07-06T20:14:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293bbff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29340b5c90 0x7f29340d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293bbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29340b6110 0x7f29340d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293bbff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340cf780 con 0x7f293415fca0 2023-07-06T20:14:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293bbff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29340cd490 con 0x7f29340b5c90 2023-07-06T20:14:20.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293bbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340cd770 con 0x7f29340b6110 2023-07-06T20:14:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293919a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29340b5c90 0x7f29340d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293919a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29340b5c90 0x7f29340d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45096/0 (socket says 172.21.15.27:45096) 2023-07-06T20:14:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293919a700 1 -- 172.21.15.27:0/1844861192 learned_addr learned my addr 172.21.15.27:0/1844861192 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:20.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.231+0000 7f293a19c700 1 --2- 172.21.15.27:0/1844861192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29340b6110 0x7f29340d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f293919a700 1 -- 172.21.15.27:0/1844861192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29340b6110 msgr2=0x7f29340d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f293919a700 1 --2- 172.21.15.27:0/1844861192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29340b6110 0x7f29340d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f293919a700 1 -- 172.21.15.27:0/1844861192 >> v1:172.21.15.153:6789/0 conn(0x7f293415fca0 legacy=0x7f29340d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f293919a700 1 -- 172.21.15.27:0/1844861192 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340cda50 con 0x7f29340b5c90 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f2938999700 1 -- 172.21.15.27:0/1844861192 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765406329 0 0) 0x7f29340cf780 con 0x7f293415fca0 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f293919a700 1 --2- 172.21.15.27:0/1844861192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340d2d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f29240098f0 tx=0x7f2924009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26dccf2a3a00966 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f2938999700 1 -- 172.21.15.27:0/1844861192 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924011830 con 0x7f29340b5c90 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f2938999700 1 -- 172.21.15.27:0/1844861192 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2924011990 con 0x7f29340b5c90 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.232+0000 7f2938999700 1 -- 172.21.15.27:0/1844861192 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924011d20 con 0x7f29340b5c90 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 -- 172.21.15.27:0/1844861192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 msgr2=0x7f29340d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 --2- 172.21.15.27:0/1844861192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340d2d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f29240098f0 tx=0x7f2924009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 -- 172.21.15.27:0/1844861192 shutdown_connections 2023-07-06T20:14:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 --2- 172.21.15.27:0/1844861192 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29340b6110 0x7f29340d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 --2- 172.21.15.27:0/1844861192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340d2d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 -- 172.21.15.27:0/1844861192 >> 172.21.15.27:0/1844861192 conn(0x7f293406c150 msgr2=0x7f293406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 -- 172.21.15.27:0/1844861192 shutdown_connections 2023-07-06T20:14:20.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.233+0000 7f293bbff700 1 -- 172.21.15.27:0/1844861192 wait complete. 2023-07-06T20:14:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.235+0000 7f293bbff700 1 Processor -- start 2023-07-06T20:14:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.235+0000 7f293bbff700 1 -- start start 2023-07-06T20:14:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293bbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293bbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29340b6110 0x7f29341f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293bbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f293415fca0 0x7f29340b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293bbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29341f1be0 con 0x7f29340b5c90 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293bbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29341f56c0 con 0x7f29340b6110 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293bbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2934157310 con 0x7f293415fca0 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45108/0 (socket says 172.21.15.27:45108) 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 -- 172.21.15.27:0/208382810 learned_addr learned my addr 172.21.15.27:0/208382810 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 -- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f293415fca0 msgr2=0x7f29340b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f293415fca0 0x7f29340b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 -- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29340b6110 msgr2=0x7f29341f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29340b6110 0x7f29341f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.236+0000 7f293999b700 1 -- 172.21.15.27:0/208382810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924009550 con 0x7f29340b5c90 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.237+0000 7f293999b700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340c9210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f293000c4d0 tx=0x7f293000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.237+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29300054e0 con 0x7f29340b5c90 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.237+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2930003680 con 0x7f29340b5c90 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.237+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930003a30 con 0x7f29340b5c90 2023-07-06T20:14:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.237+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2934156d50 con 0x7f29340b5c90 2023-07-06T20:14:20.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.237+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29341f52e0 con 0x7f29340b5c90 2023-07-06T20:14:20.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.238+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29341f5800 con 0x7f29340b5c90 2023-07-06T20:14:20.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.239+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2930003b90 con 0x7f29340b5c90 2023-07-06T20:14:20.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.240+0000 7f292a7fc700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f291806ace0 0x7f291806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.240+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29300888e0 con 0x7f29340b5c90 2023-07-06T20:14:20.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.241+0000 7f293919a700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f291806ace0 0x7f291806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.241+0000 7f293919a700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f291806ace0 0x7f291806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f29341f51a0 tx=0x7f2924010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:20.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.244+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29341f5800 con 0x7f29340b5c90 2023-07-06T20:14:20.332 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:14:20.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:14:20.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.459+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29341f5800 con 0x7f29340b5c90 2023-07-06T20:14:20.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.459+0000 7f292a7fc700 1 -- 172.21.15.27:0/208382810 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2930017070 con 0x7f29340b5c90 2023-07-06T20:14:20.462 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:20.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f291806ace0 msgr2=0x7f291806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f291806ace0 0x7f291806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f29341f51a0 tx=0x7f2924010000 comp rx=0 tx=0).stop 2023-07-06T20:14:20.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 msgr2=0x7f29340c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340c9210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f293000c4d0 tx=0x7f293000c890 comp rx=0 tx=0).stop 2023-07-06T20:14:20.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 shutdown_connections 2023-07-06T20:14:20.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f291806ace0 0x7f291806d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f293415fca0 0x7f29340b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29340b5c90 0x7f29340c9210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 --2- 172.21.15.27:0/208382810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29340b6110 0x7f29341f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 >> 172.21.15.27:0/208382810 conn(0x7f293406c150 msgr2=0x7f29340d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:20.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 shutdown_connections 2023-07-06T20:14:20.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.462+0000 7f293bbff700 1 -- 172.21.15.27:0/208382810 wait complete. 2023-07-06T20:14:20.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:20.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.725+0000 7f43637ff700 1 Processor -- start 2023-07-06T20:14:20.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.725+0000 7f43637ff700 1 -- start start 2023-07-06T20:14:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f43637ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c15fca0 0x7f435c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f43637ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f435c0b5c90 0x7f435c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f43637ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c0cf780 con 0x7f435c0b6100 2023-07-06T20:14:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f43637ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c0cd490 con 0x7f435c0b5c90 2023-07-06T20:14:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f43637ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0cd770 con 0x7f435c15fca0 2023-07-06T20:14:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f4360d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f435c0b5c90 0x7f435c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f4360d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f435c0b5c90 0x7f435c0d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45120/0 (socket says 172.21.15.27:45120) 2023-07-06T20:14:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f4360d9a700 1 -- 172.21.15.27:0/766575383 learned_addr learned my addr 172.21.15.27:0/766575383 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:20.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.726+0000 7f436159b700 1 --2- 172.21.15.27:0/766575383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c15fca0 0x7f435c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4360d9a700 1 -- 172.21.15.27:0/766575383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c15fca0 msgr2=0x7f435c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:20.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4360d9a700 1 --2- 172.21.15.27:0/766575383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c15fca0 0x7f435c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4360d9a700 1 -- 172.21.15.27:0/766575383 >> v1:172.21.15.153:6789/0 conn(0x7f435c0b6100 legacy=0x7f435c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4360d9a700 1 -- 172.21.15.27:0/766575383 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0cda50 con 0x7f435c0b5c90 2023-07-06T20:14:20.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4360d9a700 1 --2- 172.21.15.27:0/766575383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c0b5c90 0x7f435c0d2d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f434c0098f0 tx=0x7f434c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aee35736e592de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4353fff700 1 -- 172.21.15.27:0/766575383 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c011830 con 0x7f435c0b5c90 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4353fff700 1 -- 172.21.15.27:0/766575383 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c011990 con 0x7f435c0b5c90 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.727+0000 7f4353fff700 1 -- 172.21.15.27:0/766575383 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c011d20 con 0x7f435c0b5c90 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 -- 172.21.15.27:0/766575383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c0b5c90 msgr2=0x7f435c0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 --2- 172.21.15.27:0/766575383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c0b5c90 0x7f435c0d2d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f434c0098f0 tx=0x7f434c009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 -- 172.21.15.27:0/766575383 shutdown_connections 2023-07-06T20:14:20.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 --2- 172.21.15.27:0/766575383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c15fca0 0x7f435c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 --2- 172.21.15.27:0/766575383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c0b5c90 0x7f435c0d2d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 -- 172.21.15.27:0/766575383 >> 172.21.15.27:0/766575383 conn(0x7f435c06c150 msgr2=0x7f435c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:20.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 -- 172.21.15.27:0/766575383 shutdown_connections 2023-07-06T20:14:20.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.728+0000 7f43637ff700 1 -- 172.21.15.27:0/766575383 wait complete. 2023-07-06T20:14:20.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.730+0000 7f43637ff700 1 Processor -- start 2023-07-06T20:14:20.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.730+0000 7f43637ff700 1 -- start start 2023-07-06T20:14:20.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f43637ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435c0b5c90 0x7f435c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f43637ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c0b6100 0x7f435c0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f43637ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 0x7f435c0b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f43637ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1f3210 con 0x7f435c15fca0 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f43637ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1f3490 con 0x7f435c0b5c90 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f43637ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1f3770 con 0x7f435c0b6100 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f4361d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 0x7f435c0b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f4361d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 0x7f435c0b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45128/0 (socket says 172.21.15.27:45128) 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f4361d9c700 1 -- 172.21.15.27:0/2736281614 learned_addr learned my addr 172.21.15.27:0/2736281614 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f4360d9a700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c0b6100 0x7f435c0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.731+0000 7f436159b700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435c0b5c90 0x7f435c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4361d9c700 1 -- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c0b6100 msgr2=0x7f435c0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4361d9c700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c0b6100 0x7f435c0b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4361d9c700 1 -- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435c0b5c90 msgr2=0x7f435c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4361d9c700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435c0b5c90 0x7f435c1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4361d9c700 1 -- 172.21.15.27:0/2736281614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c009550 con 0x7f435c15fca0 2023-07-06T20:14:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4361d9c700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 0x7f435c0b80b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f435400cbc0 tx=0x7f435400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354003c00 con 0x7f435c15fca0 2023-07-06T20:14:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4354003d60 con 0x7f435c15fca0 2023-07-06T20:14:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400aae0 con 0x7f435c15fca0 2023-07-06T20:14:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c1f3a50 con 0x7f435c15fca0 2023-07-06T20:14:20.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.732+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c157730 con 0x7f435c15fca0 2023-07-06T20:14:20.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.734+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c1f3f70 con 0x7f435c15fca0 2023-07-06T20:14:20.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.734+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f435400ac40 con 0x7f435c15fca0 2023-07-06T20:14:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.735+0000 7f4351ffb700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f434006ace0 0x7f434006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.736+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4354089430 con 0x7f435c15fca0 2023-07-06T20:14:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.736+0000 7f436159b700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f434006ace0 0x7f434006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.737+0000 7f436159b700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f434006ace0 0x7f434006d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4358009b40 tx=0x7f4358009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.740+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f43540544e0 con 0x7f435c15fca0 2023-07-06T20:14:20.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.946+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f435c0008d0 con 0x7f435c15fca0 2023-07-06T20:14:20.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.946+0000 7f4351ffb700 1 -- 172.21.15.27:0/2736281614 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4354057b00 con 0x7f435c15fca0 2023-07-06T20:14:20.948 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:20.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.948+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f434006ace0 msgr2=0x7f434006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.948+0000 7f43637ff700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f434006ace0 0x7f434006d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4358009b40 tx=0x7f4358009210 comp rx=0 tx=0).stop 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.948+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 msgr2=0x7f435c0b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.948+0000 7f43637ff700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 0x7f435c0b80b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f435400cbc0 tx=0x7f435400ced0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 shutdown_connections 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f434006ace0 0x7f434006d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f435c15fca0 0x7f435c0b80b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f435c0b5c90 0x7f435c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 --2- 172.21.15.27:0/2736281614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f435c0b6100 0x7f435c0b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 >> 172.21.15.27:0/2736281614 conn(0x7f435c06c150 msgr2=0x7f435c0d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 shutdown_connections 2023-07-06T20:14:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:20.949+0000 7f43637ff700 1 -- 172.21.15.27:0/2736281614 wait complete. 2023-07-06T20:14:20.969 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:14:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.170+0000 7fba9d5ff700 1 Processor -- start 2023-07-06T20:14:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.170+0000 7fba9d5ff700 1 -- start start 2023-07-06T20:14:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.171+0000 7fba9d5ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba9815fca0 0x7fba980d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.171+0000 7fba9d5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba980b5c90 0x7fba980d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.171+0000 7fba9d5ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba980cf780 con 0x7fba980b60e0 2023-07-06T20:14:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.171+0000 7fba9d5ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980cd490 con 0x7fba9815fca0 2023-07-06T20:14:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.171+0000 7fba9d5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cd770 con 0x7fba980b5c90 2023-07-06T20:14:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba977fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fba980b60e0 0x7fba980cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36120/0 (socket says 172.21.15.27:36120) 2023-07-06T20:14:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba977fe700 1 -- 172.21.15.27:0/995527306 learned_addr learned my addr 172.21.15.27:0/995527306 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba96ffd700 1 --2- 172.21.15.27:0/995527306 >> v2:172.21.15.153:3300/0 conn(0x7fba9815fca0 0x7fba980d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba967fc700 1 --2- 172.21.15.27:0/995527306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba980b5c90 0x7fba980d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba95ffb700 1 -- 172.21.15.27:0/995527306 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720132882 0 0) 0x7fba980cf780 con 0x7fba980b60e0 2023-07-06T20:14:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba95ffb700 1 -- 172.21.15.27:0/995527306 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba800035b0 con 0x7fba980b60e0 2023-07-06T20:14:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba967fc700 1 -- 172.21.15.27:0/995527306 >> v2:172.21.15.153:3300/0 conn(0x7fba9815fca0 msgr2=0x7fba980d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba967fc700 1 --2- 172.21.15.27:0/995527306 >> v2:172.21.15.153:3300/0 conn(0x7fba9815fca0 0x7fba980d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba967fc700 1 -- 172.21.15.27:0/995527306 >> v1:172.21.15.27:6789/0 conn(0x7fba980b60e0 legacy=0x7fba980cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.172+0000 7fba967fc700 1 -- 172.21.15.27:0/995527306 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980cda50 con 0x7fba980b5c90 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.173+0000 7fba967fc700 1 --2- 172.21.15.27:0/995527306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba980b5c90 0x7fba980d2d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009840 tx=0x7fba8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82295311c576489d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.173+0000 7fba95ffb700 1 -- 172.21.15.27:0/995527306 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c011830 con 0x7fba980b5c90 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.173+0000 7fba95ffb700 1 -- 172.21.15.27:0/995527306 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c011990 con 0x7fba980b5c90 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.173+0000 7fba95ffb700 1 -- 172.21.15.27:0/995527306 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c011d20 con 0x7fba980b5c90 2023-07-06T20:14:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.173+0000 7fba9d5ff700 1 -- 172.21.15.27:0/995527306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba980b5c90 msgr2=0x7fba980d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.173+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/995527306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba980b5c90 0x7fba980d2d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009840 tx=0x7fba8c009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.174+0000 7fba9d5ff700 1 -- 172.21.15.27:0/995527306 shutdown_connections 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.174+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/995527306 >> v2:172.21.15.153:3300/0 conn(0x7fba9815fca0 0x7fba980d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.174+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/995527306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba980b5c90 0x7fba980d2d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.174+0000 7fba9d5ff700 1 -- 172.21.15.27:0/995527306 >> 172.21.15.27:0/995527306 conn(0x7fba9806c150 msgr2=0x7fba9806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.174+0000 7fba9d5ff700 1 -- 172.21.15.27:0/995527306 shutdown_connections 2023-07-06T20:14:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.174+0000 7fba9d5ff700 1 -- 172.21.15.27:0/995527306 wait complete. 2023-07-06T20:14:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.176+0000 7fba9d5ff700 1 Processor -- start 2023-07-06T20:14:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.176+0000 7fba9d5ff700 1 -- start start 2023-07-06T20:14:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba9d5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 0x7fba980a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba9d5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba980b60e0 0x7fba9814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba9d5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9815fca0 0x7fba9815c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba9d5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9814c8b0 con 0x7fba980b5c90 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba9d5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9815cda0 con 0x7fba980b60e0 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba9d5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9815d040 con 0x7fba9815fca0 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 0x7fba980a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 0x7fba980a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45138/0 (socket says 172.21.15.27:45138) 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 -- 172.21.15.27:0/273284298 learned_addr learned my addr 172.21.15.27:0/273284298 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba967fc700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba980b60e0 0x7fba9814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 -- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9815fca0 msgr2=0x7fba9815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba977fe700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9815fca0 0x7fba9815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9815fca0 0x7fba9815c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 -- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba980b60e0 msgr2=0x7fba9814c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba980b60e0 0x7fba9814c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba96ffd700 1 -- 172.21.15.27:0/273284298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0094a0 con 0x7fba980b5c90 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba967fc700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba980b60e0 0x7fba9814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:21.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.177+0000 7fba977fe700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9815fca0 0x7fba9815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:21.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.178+0000 7fba96ffd700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 0x7fba980a99e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba8800e870 tx=0x7fba8800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:21.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.178+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88009590 con 0x7fba980b5c90 2023-07-06T20:14:21.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.178+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8800ee70 con 0x7fba980b5c90 2023-07-06T20:14:21.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.178+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8800ca50 con 0x7fba980b5c90 2023-07-06T20:14:21.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.178+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9815d320 con 0x7fba980b5c90 2023-07-06T20:14:21.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.178+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9815d8e0 con 0x7fba980b5c90 2023-07-06T20:14:21.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.179+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba980008d0 con 0x7fba980b5c90 2023-07-06T20:14:21.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.180+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba8800cbb0 con 0x7fba980b5c90 2023-07-06T20:14:21.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.182+0000 7fba7f7fe700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8406ace0 0x7fba8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.182+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba88089520 con 0x7fba980b5c90 2023-07-06T20:14:21.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.182+0000 7fba967fc700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8406ace0 0x7fba8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.183+0000 7fba967fc700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8406ace0 0x7fba8406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba980b2810 tx=0x7fba8c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:21.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.186+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba88057fd0 con 0x7fba980b5c90 2023-07-06T20:14:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.439+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba9815e8b0 con 0x7fba980b5c90 2023-07-06T20:14:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.440+0000 7fba7f7fe700 1 -- 172.21.15.27:0/273284298 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7fba8800ce60 con 0x7fba980b5c90 2023-07-06T20:14:21.442 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:21.442 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:14:21.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8406ace0 msgr2=0x7fba8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8406ace0 0x7fba8406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba980b2810 tx=0x7fba8c008030 comp rx=0 tx=0).stop 2023-07-06T20:14:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 msgr2=0x7fba980a99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 0x7fba980a99e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fba8800e870 tx=0x7fba8800ec30 comp rx=0 tx=0).stop 2023-07-06T20:14:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 shutdown_connections 2023-07-06T20:14:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8406ace0 0x7fba8406d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9815fca0 0x7fba9815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba980b5c90 0x7fba980a99e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 --2- 172.21.15.27:0/273284298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba980b60e0 0x7fba9814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 >> 172.21.15.27:0/273284298 conn(0x7fba9806c150 msgr2=0x7fba980cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:21.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.442+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 shutdown_connections 2023-07-06T20:14:21.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.443+0000 7fba9d5ff700 1 -- 172.21.15.27:0/273284298 wait complete. 2023-07-06T20:14:21.446 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:14:21.462 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:21.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.663+0000 7f32265ff700 1 Processor -- start 2023-07-06T20:14:21.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.663+0000 7f32265ff700 1 -- start start 2023-07-06T20:14:21.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f32265ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f32200b6110 0x7f32200cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f32265ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200cf780 con 0x7f32200b5c90 2023-07-06T20:14:21.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f32265ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200cd490 con 0x7f322015fca0 2023-07-06T20:14:21.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f32265ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200cd770 con 0x7f32200b6110 2023-07-06T20:14:21.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f321f7fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f32200b5c90 0x7f32200d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36126/0 (socket says 172.21.15.27:36126) 2023-07-06T20:14:21.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f321f7fe700 1 -- 172.21.15.27:0/2716623677 learned_addr learned my addr 172.21.15.27:0/2716623677 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:21.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f3224b9c700 1 --2- 172.21.15.27:0/2716623677 >> v2:172.21.15.27:3300/0 conn(0x7f32200b6110 0x7f32200cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f3224b9c700 1 -- 172.21.15.27:0/2716623677 >> v1:172.21.15.153:6789/0 conn(0x7f322015fca0 legacy=0x7f32200d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f3224b9c700 1 -- 172.21.15.27:0/2716623677 >> v1:172.21.15.27:6789/0 conn(0x7f32200b5c90 legacy=0x7f32200d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f3224b9c700 1 -- 172.21.15.27:0/2716623677 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200cda50 con 0x7f32200b6110 2023-07-06T20:14:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.665+0000 7f321effd700 1 -- 172.21.15.27:0/2716623677 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096893353 0 0) 0x7f32200cf780 con 0x7f32200b5c90 2023-07-06T20:14:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.666+0000 7f3224b9c700 1 --2- 172.21.15.27:0/2716623677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200cfc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3218009d90 tx=0x7f321800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fccc6853ac877d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.666+0000 7f321effd700 1 -- 172.21.15.27:0/2716623677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321800e070 con 0x7f32200b6110 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.666+0000 7f321effd700 1 -- 172.21.15.27:0/2716623677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321800bbd0 con 0x7f32200b6110 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.666+0000 7f321effd700 1 -- 172.21.15.27:0/2716623677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218012430 con 0x7f32200b6110 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.666+0000 7f32265ff700 1 -- 172.21.15.27:0/2716623677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 msgr2=0x7f32200cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.666+0000 7f32265ff700 1 --2- 172.21.15.27:0/2716623677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200cfc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3218009d90 tx=0x7f321800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.667+0000 7f32265ff700 1 -- 172.21.15.27:0/2716623677 shutdown_connections 2023-07-06T20:14:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.667+0000 7f32265ff700 1 --2- 172.21.15.27:0/2716623677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200cfc10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.667+0000 7f32265ff700 1 -- 172.21.15.27:0/2716623677 >> 172.21.15.27:0/2716623677 conn(0x7f322006c150 msgr2=0x7f322006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.667+0000 7f32265ff700 1 -- 172.21.15.27:0/2716623677 shutdown_connections 2023-07-06T20:14:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.667+0000 7f32265ff700 1 -- 172.21.15.27:0/2716623677 wait complete. 2023-07-06T20:14:21.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.668+0000 7f32265ff700 1 Processor -- start 2023-07-06T20:14:21.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.669+0000 7f32265ff700 1 -- start start 2023-07-06T20:14:21.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f32265ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32200b5c90 0x7f32200b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f32265ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f32265ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f322015fca0 0x7f3220147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f32265ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200c9190 con 0x7f32200b6110 2023-07-06T20:14:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f32265ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200c9470 con 0x7f32200b5c90 2023-07-06T20:14:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f32265ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3220144f90 con 0x7f322015fca0 2023-07-06T20:14:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45156/0 (socket says 172.21.15.27:45156) 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 -- 172.21.15.27:0/3693284449 learned_addr learned my addr 172.21.15.27:0/3693284449 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321ffff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32200b5c90 0x7f32200b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f3224b9c700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f322015fca0 0x7f3220147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 -- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f322015fca0 msgr2=0x7f3220147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f322015fca0 0x7f3220147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 -- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32200b5c90 msgr2=0x7f32200b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32200b5c90 0x7f32200b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.670+0000 7f321f7fe700 1 -- 172.21.15.27:0/3693284449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3218009a70 con 0x7f32200b6110 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.671+0000 7f321f7fe700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200b3640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f321400c520 tx=0x7f321400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.671+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214004c80 con 0x7f32200b6110 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.671+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214004e00 con 0x7f32200b6110 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.671+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3220145270 con 0x7f32200b6110 2023-07-06T20:14:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.671+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214007310 con 0x7f32200b6110 2023-07-06T20:14:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.671+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3220157ff0 con 0x7f32200b6110 2023-07-06T20:14:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.672+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32201454b0 con 0x7f32200b6110 2023-07-06T20:14:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.673+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3214007470 con 0x7f32200b6110 2023-07-06T20:14:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.674+0000 7f321cff9700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f320c06ace0 0x7f320c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.674+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3214089100 con 0x7f32200b6110 2023-07-06T20:14:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.675+0000 7f321ffff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f320c06ace0 0x7f320c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.675+0000 7f321ffff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f320c06ace0 0x7f320c06d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3210007fd0 tx=0x7f3210009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:21.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.678+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f32201454b0 con 0x7f32200b6110 2023-07-06T20:14:21.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.886+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32201454b0 con 0x7f32200b6110 2023-07-06T20:14:21.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.887+0000 7f321cff9700 1 -- 172.21.15.27:0/3693284449 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f32201454b0 con 0x7f32200b6110 2023-07-06T20:14:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:21.889 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.889+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f320c06ace0 msgr2=0x7f320c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.889+0000 7f32265ff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f320c06ace0 0x7f320c06d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3210007fd0 tx=0x7f3210009040 comp rx=0 tx=0).stop 2023-07-06T20:14:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.889+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 msgr2=0x7f32200b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.889+0000 7f32265ff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200b3640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f321400c520 tx=0x7f321400c830 comp rx=0 tx=0).stop 2023-07-06T20:14:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 shutdown_connections 2023-07-06T20:14:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f320c06ace0 0x7f320c06d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f322015fca0 0x7f3220147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32200b5c90 0x7f32200b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 --2- 172.21.15.27:0/3693284449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32200b6110 0x7f32200b3640 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 >> 172.21.15.27:0/3693284449 conn(0x7f322006c150 msgr2=0x7f32200cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 shutdown_connections 2023-07-06T20:14:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:21.890+0000 7f32265ff700 1 -- 172.21.15.27:0/3693284449 wait complete. 2023-07-06T20:14:21.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:14:22.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.112+0000 7fea2b9ff700 1 Processor -- start 2023-07-06T20:14:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.113+0000 7fea2b9ff700 1 -- start start 2023-07-06T20:14:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2b9ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fea2415fca0 0x7fea240d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2b9ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea240b5c90 0x7fea240d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea240d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2b9ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240cf780 con 0x7fea240b5c90 2023-07-06T20:14:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2b9ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240cd490 con 0x7fea2415fca0 2023-07-06T20:14:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea240cd770 con 0x7fea240b6110 2023-07-06T20:14:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea28f9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea240b5c90 0x7fea240d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea28f9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea240b5c90 0x7fea240d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39486/0 (socket says 172.21.15.27:39486) 2023-07-06T20:14:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea2979b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fea2415fca0 0x7fea240d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea28f9a700 1 -- 172.21.15.27:0/209261301 learned_addr learned my addr 172.21.15.27:0/209261301 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.114+0000 7fea29f9c700 1 --2- 172.21.15.27:0/209261301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea240d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea28f9a700 1 -- 172.21.15.27:0/209261301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 msgr2=0x7fea240d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea28f9a700 1 --2- 172.21.15.27:0/209261301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea240d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea28f9a700 1 -- 172.21.15.27:0/209261301 >> v2:172.21.15.153:3300/0 conn(0x7fea2415fca0 msgr2=0x7fea240d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea28f9a700 1 --2- 172.21.15.27:0/209261301 >> v2:172.21.15.153:3300/0 conn(0x7fea2415fca0 0x7fea240d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea28f9a700 1 -- 172.21.15.27:0/209261301 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea240cda50 con 0x7fea240b5c90 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea28f9a700 1 --2- 172.21.15.27:0/209261301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea240b5c90 0x7fea240d2d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea240d0930 tx=0x7fea14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0d3154260fe5f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:22.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea1bfff700 1 -- 172.21.15.27:0/209261301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14011830 con 0x7fea240b5c90 2023-07-06T20:14:22.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea1bfff700 1 -- 172.21.15.27:0/209261301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14011990 con 0x7fea240b5c90 2023-07-06T20:14:22.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.115+0000 7fea1bfff700 1 -- 172.21.15.27:0/209261301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14011d20 con 0x7fea240b5c90 2023-07-06T20:14:22.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 -- 172.21.15.27:0/209261301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea240b5c90 msgr2=0x7fea240d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:22.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/209261301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea240b5c90 0x7fea240d2d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea240d0930 tx=0x7fea14009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 -- 172.21.15.27:0/209261301 shutdown_connections 2023-07-06T20:14:22.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/209261301 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea240d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/209261301 >> v2:172.21.15.153:3300/0 conn(0x7fea2415fca0 0x7fea240d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/209261301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea240b5c90 0x7fea240d2d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 -- 172.21.15.27:0/209261301 >> 172.21.15.27:0/209261301 conn(0x7fea2406c150 msgr2=0x7fea2406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:22.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 -- 172.21.15.27:0/209261301 shutdown_connections 2023-07-06T20:14:22.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.116+0000 7fea2b9ff700 1 -- 172.21.15.27:0/209261301 wait complete. 2023-07-06T20:14:22.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.118+0000 7fea2b9ff700 1 Processor -- start 2023-07-06T20:14:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.118+0000 7fea2b9ff700 1 -- start start 2023-07-06T20:14:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea240b5c90 0x7fea240b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea241f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2b9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 0x7fea24157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea240b55d0 con 0x7fea2415fca0 2023-07-06T20:14:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea240b43b0 con 0x7fea240b5c90 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea240b39e0 con 0x7fea240b6110 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea29f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 0x7fea24157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea29f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 0x7fea24157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39490/0 (socket says 172.21.15.27:39490) 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea29f9c700 1 -- 172.21.15.27:0/4218965560 learned_addr learned my addr 172.21.15.27:0/4218965560 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea2979b700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea240b5c90 0x7fea240b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.119+0000 7fea28f9a700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea241f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea29f9c700 1 -- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 msgr2=0x7fea241f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:22.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea29f9c700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea241f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea29f9c700 1 -- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea240b5c90 msgr2=0x7fea240b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea29f9c700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea240b5c90 0x7fea240b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea29f9c700 1 -- 172.21.15.27:0/4218965560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea14009550 con 0x7fea2415fca0 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea29f9c700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 0x7fea24157420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00eed0 tx=0x7fea1c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c010040 con 0x7fea2415fca0 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00bc20 con 0x7fea2415fca0 2023-07-06T20:14:22.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c009e40 con 0x7fea2415fca0 2023-07-06T20:14:22.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea241f2d50 con 0x7fea2415fca0 2023-07-06T20:14:22.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.120+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea240b30b0 con 0x7fea2415fca0 2023-07-06T20:14:22.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.122+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea240b3b50 con 0x7fea2415fca0 2023-07-06T20:14:22.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.122+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea1c014400 con 0x7fea2415fca0 2023-07-06T20:14:22.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.123+0000 7fea19ffb700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fea0806ace0 0x7fea0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:22.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.123+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea1c0897c0 con 0x7fea2415fca0 2023-07-06T20:14:22.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.124+0000 7fea2979b700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fea0806ace0 0x7fea0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:22.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.124+0000 7fea2979b700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fea0806ace0 0x7fea0806d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea240b2810 tx=0x7fea20009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:22.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.127+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fea1c054860 con 0x7fea2415fca0 2023-07-06T20:14:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.377+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea240008d0 con 0x7fea2415fca0 2023-07-06T20:14:22.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.377+0000 7fea19ffb700 1 -- 172.21.15.27:0/4218965560 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7fea1c057e80 con 0x7fea2415fca0 2023-07-06T20:14:22.379 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:22.380 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:14:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fea0806ace0 msgr2=0x7fea0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fea0806ace0 0x7fea0806d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea240b2810 tx=0x7fea20009210 comp rx=0 tx=0).stop 2023-07-06T20:14:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 msgr2=0x7fea24157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 0x7fea24157420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00eed0 tx=0x7fea1c00b390 comp rx=0 tx=0).stop 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 shutdown_connections 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fea0806ace0 0x7fea0806d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2415fca0 0x7fea24157420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea240b5c90 0x7fea240b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 --2- 172.21.15.27:0/4218965560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea240b6110 0x7fea241f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 >> 172.21.15.27:0/4218965560 conn(0x7fea2406c150 msgr2=0x7fea240d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 shutdown_connections 2023-07-06T20:14:22.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:22.380+0000 7fea2b9ff700 1 -- 172.21.15.27:0/4218965560 wait complete. 2023-07-06T20:14:22.384 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:14:22.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T20:13:59.519602+0000', 'modified': '2023-07-06T20:14:03.050155+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6834/3572656483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 3572656483}, {'type': 'v1', 'addr': '172.21.15.27:6834', 'nonce': 3572656483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T20:14:22.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T20:14:22.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-06T20:14:23.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.009+0000 7f9ed45ff700 1 Processor -- start 2023-07-06T20:14:23.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.010+0000 7f9ed45ff700 1 -- start start 2023-07-06T20:14:23.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed45ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed45ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ecc0b60e0 0x7f9ecc0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed45ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0cf780 con 0x7f9ecc15fca0 2023-07-06T20:14:23.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed45ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc0cd490 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed45ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0cd770 con 0x7f9ecc0b5c90 2023-07-06T20:14:23.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed2b9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ecc0b60e0 0x7f9ecc0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed2b9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ecc0b60e0 0x7f9ecc0d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39502/0 (socket says 172.21.15.27:39502) 2023-07-06T20:14:23.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed2b9c700 1 -- 172.21.15.27:0/1575336403 learned_addr learned my addr 172.21.15.27:0/1575336403 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.011+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed2b9c700 1 -- 172.21.15.27:0/1575336403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 msgr2=0x7f9ecc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed2b9c700 1 -- 172.21.15.27:0/1575336403 >> v1:172.21.15.153:6789/0 conn(0x7f9ecc15fca0 legacy=0x7f9ecc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed2b9c700 1 -- 172.21.15.27:0/1575336403 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0cda50 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d3280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009e50 tx=0x7f9ec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b484c2381dcdc75 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed1399700 1 -- 172.21.15.27:0/1575336403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400e070 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed1399700 1 -- 172.21.15.27:0/1575336403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec400b9c0 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.012+0000 7f9ed1399700 1 -- 172.21.15.27:0/1575336403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400bd70 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1575336403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 msgr2=0x7f9ecc0d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d3280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009e50 tx=0x7f9ec400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1575336403 shutdown_connections 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d3280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1575336403 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1575336403 >> 172.21.15.27:0/1575336403 conn(0x7f9ecc06c150 msgr2=0x7f9ecc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1575336403 shutdown_connections 2023-07-06T20:14:23.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.013+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1575336403 wait complete. 2023-07-06T20:14:23.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.015+0000 7f9ed45ff700 1 Processor -- start 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.015+0000 7f9ed45ff700 1 -- start start 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed45ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed45ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed45ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ecc15fca0 0x7f9ecc14c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed45ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc15de00 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed45ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc15d1f0 con 0x7f9ecc15fca0 2023-07-06T20:14:23.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed45ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc15d4d0 con 0x7f9ecc0b5c90 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39514/0 (socket says 172.21.15.27:39514) 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/1666243738 learned_addr learned my addr 172.21.15.27:0/1666243738 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed239b700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ecc15fca0 0x7f9ecc14c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 msgr2=0x7f9ecc15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ecc15fca0 msgr2=0x7f9ecc14c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ecc15fca0 0x7f9ecc14c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.016+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/1666243738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4009b30 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.017+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d2c50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00e810 tx=0x7f9ebc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.017+0000 7f9ec2ffd700 1 -- 172.21.15.27:0/1666243738 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00c070 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.017+0000 7f9ec2ffd700 1 -- 172.21.15.27:0/1666243738 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc004230 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.017+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc15d7b0 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.017+0000 7f9ec2ffd700 1 -- 172.21.15.27:0/1666243738 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0036a0 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.017+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc15e7d0 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.020+0000 7f9ec2ffd700 1 -- 172.21.15.27:0/1666243738 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ebc003800 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.021+0000 7f9ec2ffd700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9eb006ace0 0x7f9eb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.021+0000 7f9ec2ffd700 1 -- 172.21.15.27:0/1666243738 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ebc0891f0 con 0x7f9ecc0b60e0 2023-07-06T20:14:23.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.021+0000 7f9ed239b700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9eb006ace0 0x7f9eb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.022+0000 7f9ed239b700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9eb006ace0 0x7f9eb006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8007fd0 tx=0x7f9ec8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.023+0000 7f9ee7e80b80 1 Processor -- start 2023-07-06T20:14:23.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.023+0000 7f9ee7e80b80 1 -- start start 2023-07-06T20:14:23.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdaed50 0x55bf5cdaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cd8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cd00ea0 con 0x55bf5cdb4190 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cc74600 con 0x55bf5cdaed50 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cce3910 con 0x55bf5cd89b50 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39530/0 (socket says 172.21.15.27:39530) 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.024+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/807765003 learned_addr learned my addr 172.21.15.27:0/807765003 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed239b700 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdaed50 0x55bf5cdaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cd8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 msgr2=0x55bf5cd8c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cd8c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdaed50 msgr2=0x55bf5cdaf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdaed50 0x55bf5cdaf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/807765003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf5cad0790 con 0x55bf5cdb4190 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd2a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009600 tx=0x7f9ebc010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfcfd9580e72ae7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9eaffff700 1 -- 172.21.15.27:0/807765003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc060210 con 0x55bf5cdb4190 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9eaffff700 1 -- 172.21.15.27:0/807765003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc056260 con 0x55bf5cdb4190 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.025+0000 7f9eaffff700 1 -- 172.21.15.27:0/807765003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0573b0 con 0x55bf5cdb4190 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/807765003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 msgr2=0x55bf5cdd2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd2a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009600 tx=0x7f9ebc010e20 comp rx=0 tx=0).stop 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/807765003 shutdown_connections 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cd8c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdaed50 0x55bf5cdaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/807765003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd2a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/807765003 >> 172.21.15.27:0/807765003 conn(0x55bf5cd81310 msgr2=0x55bf5cdd15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/807765003 shutdown_connections 2023-07-06T20:14:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.026+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/807765003 wait complete. 2023-07-06T20:14:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.027+0000 7f9ee7e80b80 1 -- start start 2023-07-06T20:14:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.027+0000 7f9ee7e80b80 1 Processor -- start 2023-07-06T20:14:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 0x55bf5cdc9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdaed50 0x55bf5cda6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cda6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5ccdcbe0 con 0x55bf5cd89b50 2023-07-06T20:14:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cc9a5b0 con 0x55bf5cdb4190 2023-07-06T20:14:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cc80910 con 0x55bf5cdaed50 2023-07-06T20:14:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.028+0000 7f9ee7e80b80 10 client.0 _ll_register_callbacks cb 0x55bf5cc98b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:14:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 0x55bf5cdc9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 0x55bf5cdc9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39532/0 (socket says 172.21.15.27:39532) 2023-07-06T20:14:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 -- 172.21.15.27:0/3326965597 learned_addr learned my addr 172.21.15.27:0/3326965597 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdaed50 0x55bf5cda6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cda6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 -- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdaed50 msgr2=0x55bf5cda6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdaed50 0x55bf5cda6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 -- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 msgr2=0x55bf5cda6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cda6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 -- 172.21.15.27:0/3326965597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc15e7d0 con 0x55bf5cd89b50 2023-07-06T20:14:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cda6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ed239b700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 0x55bf5cdc9150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8010010 tx=0x7f9ec8001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb64cbc4bf6b835b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.029+0000 7f9ead7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ead7fa700 1 -- 172.21.15.27:0/3326965597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8014ba0 con 0x55bf5cd89b50 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ead7fa700 1 -- 172.21.15.27:0/3326965597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ec801c040 con 0x55bf5cd89b50 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf5cd7e920 con 0x55bf5cd89b50 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ead7fa700 1 -- 172.21.15.27:0/3326965597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ec8014d00 con 0x55bf5cd89b50 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 10 client.4686 fetch_fsmap learned FSMap version 7 2023-07-06T20:14:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55bf5cbbd0a0 con 0x55bf5cd89b50 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ead7fa700 1 -- 172.21.15.27:0/3326965597 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf5cd7e920 con 0x55bf5cd89b50 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ead7fa700 1 -- 172.21.15.27:0/3326965597 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9ec8021400 con 0x55bf5cd89b50 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 10 client.4686 fetch_fsmap finished waiting for FSMap version 7 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 10 client.4686 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 20 client.4686 populate_metadata read hostname 'smithi027' 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5cd9e3e0 0x55bf5cdee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 4 client.4686 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.030+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 --> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf5cd5b0d0 con 0x55bf5cd9e3e0 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.031+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5cd9e3e0 0x55bf5cdee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.031+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5cd9e3e0 0x55bf5cdee320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea475b873c7a4c78 server_cookie=2517bb73ddc8f33b in_seq=0 out_seq=0 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.031+0000 7f9ead7fa700 10 client.4686 ms_handle_connect on v2:172.21.15.27:6832/3572656483 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ead7fa700 1 -- 172.21.15.27:0/3326965597 <== mds.0 v2:172.21.15.27:6832/3572656483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf5cd5b0d0 con 0x55bf5cd9e3e0 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ead7fa700 10 client.4686 handle_command_reply: tid=0 2023-07-06T20:14:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 client.4686 shutdown 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 msgr2=0x55bf5cdc9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 0x55bf5cdc9150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8010010 tx=0x7f9ec8001080 comp rx=0 tx=0).stop 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 shutdown_connections 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5cd9e3e0 0x55bf5cdee320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cda6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cd89b50 0x55bf5cdc9150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/3326965597 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdaed50 0x55bf5cda6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 >> 172.21.15.27:0/3326965597 conn(0x55bf5cd81310 msgr2=0x55bf5cdd1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ead7fa700 0 client.4686 ms_handle_reset on v2:172.21.15.27:6832/3572656483 2023-07-06T20:14:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 shutdown_connections 2023-07-06T20:14:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.032+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/3326965597 wait complete. 2023-07-06T20:14:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.033+0000 7f9ee7e80b80 20 client.4686 trim_cache size 0 max 16384 2023-07-06T20:14:23.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.051+0000 7f9ee7e80b80 1 Processor -- start 2023-07-06T20:14:23.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.051+0000 7f9ee7e80b80 1 -- start start 2023-07-06T20:14:23.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cda7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cda8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cad1390 con 0x55bf5cdaed50 2023-07-06T20:14:23.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cd7e460 con 0x55bf5cdb4190 2023-07-06T20:14:23.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cdc8390 con 0x55bf5cd89b50 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cda8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cda8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39536/0 (socket says 172.21.15.27:39536) 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/4183189396 learned_addr learned my addr 172.21.15.27:0/4183189396 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed239b700 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cda7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 msgr2=0x55bf5cda7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cda7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 msgr2=0x55bf5cdd1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/4183189396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf5cca2b10 con 0x55bf5cdaed50 2023-07-06T20:14:23.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.052+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cda8220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc05c040 tx=0x7f9ebc010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424c5290b85119df server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ec0bff700 1 -- 172.21.15.27:0/4183189396 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc060290 con 0x55bf5cdaed50 2023-07-06T20:14:23.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ec0bff700 1 -- 172.21.15.27:0/4183189396 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc0545d0 con 0x55bf5cdaed50 2023-07-06T20:14:23.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ec0bff700 1 -- 172.21.15.27:0/4183189396 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc076e00 con 0x55bf5cdaed50 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/4183189396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 msgr2=0x55bf5cda8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cda8220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc05c040 tx=0x7f9ebc010f90 comp rx=0 tx=0).stop 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/4183189396 shutdown_connections 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cdb4190 0x55bf5cdd1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cd89b50 0x55bf5cda7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/4183189396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cda8220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/4183189396 >> 172.21.15.27:0/4183189396 conn(0x55bf5cd81310 msgr2=0x55bf5cd86dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/4183189396 shutdown_connections 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.053+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/4183189396 wait complete. 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.054+0000 7f9ee7e80b80 1 -- start start 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.054+0000 7f9ee7e80b80 1 Processor -- start 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cd89b50 0x55bf5cda57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cd8a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdb4190 0x55bf5cd8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cda6ff0 con 0x55bf5cdaed50 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cda6060 con 0x55bf5cd89b50 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5cdb5fa0 con 0x55bf5cdb4190 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ee7e80b80 10 client.0 _ll_register_callbacks cb 0x7f9ec800fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cd8a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ed1b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cd8a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39550/0 (socket says 172.21.15.27:39550) 2023-07-06T20:14:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.055+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/692743579 learned_addr learned my addr 172.21.15.27:0/692743579 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed2b9c700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdb4190 0x55bf5cd8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed239b700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cd89b50 0x55bf5cda57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdb4190 msgr2=0x55bf5cd8ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdb4190 0x55bf5cd8ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cd89b50 msgr2=0x55bf5cda57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cd89b50 0x55bf5cda57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed1b9a700 1 -- 172.21.15.27:0/692743579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf5cad0790 con 0x55bf5cdaed50 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed239b700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cd89b50 0x55bf5cda57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ed1b9a700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cd8a710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc05c040 tx=0x7f9ebc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de48266f2c9bd45 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ead7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ead7fa700 1 -- 172.21.15.27:0/692743579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc076de0 con 0x55bf5cdaed50 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ead7fa700 1 -- 172.21.15.27:0/692743579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ebc060210 con 0x55bf5cdaed50 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ebc056260 con 0x55bf5cdaed50 2023-07-06T20:14:23.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ead7fa700 1 -- 172.21.15.27:0/692743579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ebc0573b0 con 0x55bf5cdaed50 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ee7e80b80 10 client.4692 fetch_fsmap learned FSMap version 7 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.056+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55bf5cb311d0 con 0x55bf5cdaed50 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ead7fa700 1 -- 172.21.15.27:0/692743579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc056260 con 0x55bf5cdaed50 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ead7fa700 1 -- 172.21.15.27:0/692743579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9ebc05fd80 con 0x55bf5cdaed50 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ee7e80b80 10 client.4692 fetch_fsmap finished waiting for FSMap version 7 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ee7e80b80 10 client.4692 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T20:14:23.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ee7e80b80 20 client.4692 populate_metadata read hostname 'smithi027' 2023-07-06T20:14:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5ce37630 0x55bf5cdf2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ee7e80b80 4 client.4692 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-07-06T20:14:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 --> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf5ccd5d20 con 0x55bf5ce37630 2023-07-06T20:14:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ed239b700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5ce37630 0x55bf5cdf2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ed239b700 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5ce37630 0x55bf5cdf2340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba349602bc005cf server_cookie=2f5eb25ba6a514d6 in_seq=0 out_seq=0 2023-07-06T20:14:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.057+0000 7f9ead7fa700 10 client.4692 ms_handle_connect on v2:172.21.15.27:6832/3572656483 2023-07-06T20:14:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ead7fa700 1 -- 172.21.15.27:0/692743579 <== mds.0 v2:172.21.15.27:6832/3572656483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55bf5ccd5d20 con 0x55bf5ce37630 2023-07-06T20:14:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ead7fa700 10 client.4692 handle_command_reply: tid=0 2023-07-06T20:14:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 client.4692 shutdown 2023-07-06T20:14:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 msgr2=0x55bf5cd8a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cd8a710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc05c040 tx=0x7f9ebc004750 comp rx=0 tx=0).stop 2023-07-06T20:14:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 shutdown_connections 2023-07-06T20:14:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:6832/3572656483,v1:172.21.15.27:6834/3572656483] conn(0x55bf5ce37630 0x55bf5cdf2340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55bf5cdb4190 0x55bf5cd8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf5cd89b50 0x55bf5cda57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 --2- 172.21.15.27:0/692743579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bf5cdaed50 0x55bf5cd8a710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ead7fa700 0 client.4692 ms_handle_reset on v2:172.21.15.27:6832/3572656483 2023-07-06T20:14:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 >> 172.21.15.27:0/692743579 conn(0x55bf5cd81310 msgr2=0x55bf5cdcace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 shutdown_connections 2023-07-06T20:14:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 1 -- 172.21.15.27:0/692743579 wait complete. 2023-07-06T20:14:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.058+0000 7f9ee7e80b80 20 client.4692 trim_cache size 0 max 16384 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c2a9da8e-f30b-43f7-b235-3303ae838054", 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4299, 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-07-06T20:14:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-07-06T20:14:23.071 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 23.033483319999998, 2023-07-06T20:14:23.071 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 7, 2023-07-06T20:14:23.071 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 23, 2023-07-06T20:14:23.071 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 23, 2023-07-06T20:14:23.071 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 29.890325885999999 2023-07-06T20:14:23.071 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:14:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9eb006ace0 msgr2=0x7f9eb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9eb006ace0 0x7f9eb006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8007fd0 tx=0x7f9ec8009040 comp rx=0 tx=0).stop 2023-07-06T20:14:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 msgr2=0x7f9ecc0d2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d2c50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00e810 tx=0x7f9ebc00eb20 comp rx=0 tx=0).stop 2023-07-06T20:14:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 shutdown_connections 2023-07-06T20:14:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9eb006ace0 0x7f9eb006d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ecc15fca0 0x7f9ecc14c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ecc0b5c90 0x7f9ecc15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 --2- 172.21.15.27:0/1666243738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ecc0b60e0 0x7f9ecc0d2c50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 >> 172.21.15.27:0/1666243738 conn(0x7f9ecc06c150 msgr2=0x7f9ecc15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 shutdown_connections 2023-07-06T20:14:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.059+0000 7f9ed45ff700 1 -- 172.21.15.27:0/1666243738 wait complete. 2023-07-06T20:14:23.079 INFO:teuthology.run_tasks:Running task kclient... 2023-07-06T20:14:23.102 INFO:tasks.kclient:Mounting kernel clients... 2023-07-06T20:14:23.102 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-06T20:14:23.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.103 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:14:23.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.150 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-07-06T20:14:23.225 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2023-07-06T20:14:23.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:14:23.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.229 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:14:23.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-07-06T20:14:23.348 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2023-07-06T20:14:23.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:14:23.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.352 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:14:23.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.404 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-07-06T20:14:23.480 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-07-06T20:14:23.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:14:23.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.484 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:14:23.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:23.530 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-07-06T20:14:23.601 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-07-06T20:14:23.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:14:23.604 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-07-06T20:14:23.604 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T20:14:23.604 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T20:14:23.605 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T20:14:23.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.805+0000 7f9e817ff700 1 Processor -- start 2023-07-06T20:14:23.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.806+0000 7f9e817ff700 1 -- start start 2023-07-06T20:14:23.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e817ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e817ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9e7c0b5c90 0x7f9e7c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e817ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7c0cf780 con 0x7f9e7c0b6100 2023-07-06T20:14:23.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e817ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7c0cd490 con 0x7f9e7c0b5c90 2023-07-06T20:14:23.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e817ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0cd770 con 0x7f9e7c15fca0 2023-07-06T20:14:23.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e7b7fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9e7c0b6100 0x7f9e7c0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60962/0 (socket says 172.21.15.27:60962) 2023-07-06T20:14:23.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e7b7fe700 1 -- 172.21.15.27:0/955708546 learned_addr learned my addr 172.21.15.27:0/955708546 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:23.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e73fff700 1 --2- 172.21.15.27:0/955708546 >> v2:172.21.15.153:3300/0 conn(0x7f9e7c0b5c90 0x7f9e7c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.807+0000 7f9e7affd700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e7a7fc700 1 -- 172.21.15.27:0/955708546 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334772575 0 0) 0x7f9e7c0cf780 con 0x7f9e7c0b6100 2023-07-06T20:14:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e7a7fc700 1 -- 172.21.15.27:0/955708546 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e680035b0 con 0x7f9e7c0b6100 2023-07-06T20:14:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e73fff700 1 -- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 msgr2=0x7f9e7c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e73fff700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e73fff700 1 -- 172.21.15.27:0/955708546 >> v1:172.21.15.27:6789/0 conn(0x7f9e7c0b6100 legacy=0x7f9e7c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e73fff700 1 -- 172.21.15.27:0/955708546 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c0cda50 con 0x7f9e7c0b5c90 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e7affd700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e73fff700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c0d2d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009840 tx=0x7f9e64009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda9f433cb1b095d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e7a7fc700 1 -- 172.21.15.27:0/955708546 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64011830 con 0x7f9e7c0b5c90 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.808+0000 7f9e7a7fc700 1 -- 172.21.15.27:0/955708546 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64011990 con 0x7f9e7c0b5c90 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e7a7fc700 1 -- 172.21.15.27:0/955708546 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64011d20 con 0x7f9e7c0b5c90 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 -- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 msgr2=0x7f9e7c0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c0d2d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009840 tx=0x7f9e64009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 -- 172.21.15.27:0/955708546 shutdown_connections 2023-07-06T20:14:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 --2- 172.21.15.27:0/955708546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c0d2d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 -- 172.21.15.27:0/955708546 >> 172.21.15.27:0/955708546 conn(0x7f9e7c06c150 msgr2=0x7f9e7c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 -- 172.21.15.27:0/955708546 shutdown_connections 2023-07-06T20:14:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.809+0000 7f9e817ff700 1 -- 172.21.15.27:0/955708546 wait complete. 2023-07-06T20:14:23.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.811+0000 7f9e817ff700 1 Processor -- start 2023-07-06T20:14:23.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.811+0000 7f9e817ff700 1 -- start start 2023-07-06T20:14:23.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e817ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e817ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 0x7f9e7c0d2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e817ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e817ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c15b610 con 0x7f9e7c0b6100 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e817ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c1f1e60 con 0x7f9e7c0b5c90 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e817ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0cac40 con 0x7f9e7c15fca0 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e73fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 0x7f9e7c0d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e73fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 0x7f9e7c0d2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39564/0 (socket says 172.21.15.27:39564) 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e73fff700 1 -- 172.21.15.27:0/2821076460 learned_addr learned my addr 172.21.15.27:0/2821076460 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e7b7fe700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.812+0000 7f9e7affd700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e73fff700 1 -- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 msgr2=0x7f9e7c0d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e73fff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e73fff700 1 -- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 msgr2=0x7f9e7c14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:23.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e73fff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c14b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e73fff700 1 -- 172.21.15.27:0/2821076460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0094a0 con 0x7f9e7c0b6100 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e73fff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 0x7f9e7c0d2980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e640090b0 tx=0x7f9e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004590 con 0x7f9e7c0b6100 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6401f070 con 0x7f9e7c0b6100 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640038e0 con 0x7f9e7c0b6100 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.813+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e640094a0 con 0x7f9e7c0b6100 2023-07-06T20:14:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.814+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c15bb40 con 0x7f9e7c0b6100 2023-07-06T20:14:23.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.814+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c0b3d30 con 0x7f9e7c0b6100 2023-07-06T20:14:23.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.815+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e64003a40 con 0x7f9e7c0b6100 2023-07-06T20:14:23.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.816+0000 7f9e737fe700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5c06ace0 0x7f9e5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:23.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.816+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e64021070 con 0x7f9e7c0b6100 2023-07-06T20:14:23.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.817+0000 7f9e7affd700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5c06ace0 0x7f9e5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:23.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.817+0000 7f9e7affd700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5c06ace0 0x7f9e5c06d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009cd0 tx=0x7f9e6c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:23.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:23.821+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e64055cb0 con 0x7f9e7c0b6100 2023-07-06T20:14:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.076+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9e7c0008d0 con 0x7f9e7c0b6100 2023-07-06T20:14:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.076+0000 7f9e737fe700 1 -- 172.21.15.27:0/2821076460 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9e640592d0 con 0x7f9e7c0b6100 2023-07-06T20:14:24.078 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:14:24.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5c06ace0 msgr2=0x7f9e5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:24.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5c06ace0 0x7f9e5c06d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009cd0 tx=0x7f9e6c009210 comp rx=0 tx=0).stop 2023-07-06T20:14:24.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 msgr2=0x7f9e7c0d2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:24.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 0x7f9e7c0d2980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e640090b0 tx=0x7f9e64004750 comp rx=0 tx=0).stop 2023-07-06T20:14:24.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 shutdown_connections 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5c06ace0 0x7f9e5c06d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e7c15fca0 0x7f9e7c0d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e7c0b5c90 0x7f9e7c14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 --2- 172.21.15.27:0/2821076460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0b6100 0x7f9e7c0d2980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 >> 172.21.15.27:0/2821076460 conn(0x7f9e7c06c150 msgr2=0x7f9e7c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 shutdown_connections 2023-07-06T20:14:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:24.079+0000 7f9e817ff700 1 -- 172.21.15.27:0/2821076460 wait complete. 2023-07-06T20:14:24.099 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T20:14:24.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:14:24.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-07-06T20:14:24.100 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T20:14:24.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T20:14:24.100 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T20:14:24.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T20:14:24.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:14:24.101 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T20:14:24.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:24.101 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-07-06T20:14:24.147 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:24.148 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:14:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41946sec preferred_lft 41946sec 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-07-06T20:14:24.151 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T20:14:24.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:24.152 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:24.152 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:24.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2023-07-06T20:14:24.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2023-07-06T20:14:24.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2023-07-06T20:14:24.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T20:14:24.154 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:24.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:24.592 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:14:24.715 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-07-06T20:14:24.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:24.718 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-07-06T20:14:24.791 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-07-06T20:14:24.791 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:14:24.791 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-06T20:14:24.791 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-06T20:14:24.791 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:14:24.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:24.793 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:24.793 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:24.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-06T20:14:24.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-06T20:14:24.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-06T20:14:24.794 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:25.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:25.455 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:14:25.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:25.503 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-07-06T20:14:25.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:25.550 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:25.551 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:25.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T20:14:25.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-06T20:14:25.551 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:25.698 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-06T20:14:25.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:25.699 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:25.699 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:25.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-06T20:14:25.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T20:14:25.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-06T20:14:25.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-06T20:14:25.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-06T20:14:25.701 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:25.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:25.975 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:25.975 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:25.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-07-06T20:14:25.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T20:14:25.976 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:26.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:26.158 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T20:14:26.204 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T20:14:26.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:26.205 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T20:14:26.289 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-06T20:14:26.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:26.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:26.400 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:26.400 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:26.854 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:26.854 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:26.855 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.153:3300,172.21.15.153:3301" 2023-07-06T20:14:26.855 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@c2a9da8e-f30b-43f7-b235-3303ae838054.cephfs=/ 2023-07-06T20:14:26.855 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.153:3300/172.21.15.153:3301" will pass to kernel 2023-07-06T20:14:26.860 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:14:26.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:26.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T20:14:26.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:26.933 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:27.050 INFO:teuthology.orchestra.run.smithi027.stdout:client4701 2023-07-06T20:14:27.051 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:27.051 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:27.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:27.097 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:27.243 INFO:teuthology.orchestra.run.smithi027.stdout:client4701 2023-07-06T20:14:27.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:27.244 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:27.346 INFO:teuthology.orchestra.run.smithi027.stdout:client4701 2023-07-06T20:14:27.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:27.346 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2a9da8e-f30b-43f7-b235-3303ae838054.client4701/status') 2023-07-06T20:14:27.468 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.4701 (3)192.168.0.1:0/2647877979 2023-07-06T20:14:27.469 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2023-07-06T20:14:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2023-07-06T20:14:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2023-07-06T20:14:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:69 bytes copied, 5.8265e-05 s, 1.2 MB/s 2023-07-06T20:14:27.471 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-07-06T20:14:27.471 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T20:14:27.472 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T20:14:27.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T20:14:27.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.673+0000 7fbaa41ff700 1 Processor -- start 2023-07-06T20:14:27.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.674+0000 7fbaa41ff700 1 -- start start 2023-07-06T20:14:27.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa41ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba9c15fca0 0x7fba9c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:27.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa41ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c0b6110 0x7fba9c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:27.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa41ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9c0cf780 con 0x7fba9c0b5c90 2023-07-06T20:14:27.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa41ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9c0cd490 con 0x7fba9c15fca0 2023-07-06T20:14:27.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa41ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0cd770 con 0x7fba9c0b6110 2023-07-06T20:14:27.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa1f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba9c15fca0 0x7fba9c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:27.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa1f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba9c15fca0 0x7fba9c0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39576/0 (socket says 172.21.15.27:39576) 2023-07-06T20:14:27.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa1f9b700 1 -- 172.21.15.27:0/676907822 learned_addr learned my addr 172.21.15.27:0/676907822 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.675+0000 7fbaa279c700 1 --2- 172.21.15.27:0/676907822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c0b6110 0x7fba9c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa1f9b700 1 -- 172.21.15.27:0/676907822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c0b6110 msgr2=0x7fba9c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa1f9b700 1 --2- 172.21.15.27:0/676907822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c0b6110 0x7fba9c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa1f9b700 1 -- 172.21.15.27:0/676907822 >> v1:172.21.15.153:6789/0 conn(0x7fba9c0b5c90 legacy=0x7fba9c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa1f9b700 1 -- 172.21.15.27:0/676907822 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c0cda50 con 0x7fba9c15fca0 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa0f99700 1 -- 172.21.15.27:0/676907822 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885552350 0 0) 0x7fba9c0cf780 con 0x7fba9c0b5c90 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa1f9b700 1 --2- 172.21.15.27:0/676907822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c15fca0 0x7fba9c0d1420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba980098f0 tx=0x7fba98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4650d5c7b0f8c2e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:27.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.676+0000 7fbaa0f99700 1 -- 172.21.15.27:0/676907822 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98011830 con 0x7fba9c15fca0 2023-07-06T20:14:27.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa0f99700 1 -- 172.21.15.27:0/676907822 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba98011990 con 0x7fba9c15fca0 2023-07-06T20:14:27.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa0f99700 1 -- 172.21.15.27:0/676907822 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98011d20 con 0x7fba9c15fca0 2023-07-06T20:14:27.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 -- 172.21.15.27:0/676907822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c15fca0 msgr2=0x7fba9c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:27.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/676907822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c15fca0 0x7fba9c0d1420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba980098f0 tx=0x7fba98009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:27.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 -- 172.21.15.27:0/676907822 shutdown_connections 2023-07-06T20:14:27.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/676907822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c0b6110 0x7fba9c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/676907822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c15fca0 0x7fba9c0d1420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 -- 172.21.15.27:0/676907822 >> 172.21.15.27:0/676907822 conn(0x7fba9c06c150 msgr2=0x7fba9c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:27.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 -- 172.21.15.27:0/676907822 shutdown_connections 2023-07-06T20:14:27.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.677+0000 7fbaa41ff700 1 -- 172.21.15.27:0/676907822 wait complete. 2023-07-06T20:14:27.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.679+0000 7fbaa41ff700 1 Processor -- start 2023-07-06T20:14:27.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.679+0000 7fbaa41ff700 1 -- start start 2023-07-06T20:14:27.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa41ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba9c0b5c90 0x7fba9c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:27.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa41ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 0x7fba9c0c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:27.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa41ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c15fca0 0x7fba9c14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:27.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa41ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0ca530 con 0x7fba9c0b6110 2023-07-06T20:14:27.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa41ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0ca810 con 0x7fba9c0b5c90 2023-07-06T20:14:27.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa41ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0caaf0 con 0x7fba9c15fca0 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa179a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 0x7fba9c0c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa179a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 0x7fba9c0c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39592/0 (socket says 172.21.15.27:39592) 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.680+0000 7fbaa179a700 1 -- 172.21.15.27:0/2920950769 learned_addr learned my addr 172.21.15.27:0/2920950769 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa279c700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c15fca0 0x7fba9c14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa1f9b700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba9c0b5c90 0x7fba9c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa179a700 1 -- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c15fca0 msgr2=0x7fba9c14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa179a700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c15fca0 0x7fba9c14d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa179a700 1 -- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba9c0b5c90 msgr2=0x7fba9c0cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa179a700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba9c0b5c90 0x7fba9c0cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa179a700 1 -- 172.21.15.27:0/2920950769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba98009550 con 0x7fba9c0b6110 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa1f9b700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba9c0b5c90 0x7fba9c0cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa179a700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 0x7fba9c0c9f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0011e0 tx=0x7fba8c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0054e0 con 0x7fba9c0b6110 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c003680 con 0x7fba9c0b6110 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c0cadd0 con 0x7fba9c0b6110 2023-07-06T20:14:27.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.681+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c003a30 con 0x7fba9c0b6110 2023-07-06T20:14:27.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.682+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c14d670 con 0x7fba9c0b6110 2023-07-06T20:14:27.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.682+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c0cb2f0 con 0x7fba9c0b6110 2023-07-06T20:14:27.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.684+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba8c003b90 con 0x7fba9c0b6110 2023-07-06T20:14:27.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.685+0000 7fba927fc700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8006ace0 0x7fba8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:27.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.685+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba8c088a60 con 0x7fba9c0b6110 2023-07-06T20:14:27.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.685+0000 7fbaa1f9b700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8006ace0 0x7fba8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:27.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.686+0000 7fbaa1f9b700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8006ace0 0x7fba8006d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fba98014040 tx=0x7fba98008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:27.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.689+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba9c0cb2f0 con 0x7fba9c0b6110 2023-07-06T20:14:27.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.944+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba9c0cb2f0 con 0x7fba9c0b6110 2023-07-06T20:14:27.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.944+0000 7fba927fc700 1 -- 172.21.15.27:0/2920950769 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba9c0cb2f0 con 0x7fba9c0b6110 2023-07-06T20:14:27.946 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:14:27.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.946+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8006ace0 msgr2=0x7fba8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.946+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8006ace0 0x7fba8006d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fba98014040 tx=0x7fba98008030 comp rx=0 tx=0).stop 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.946+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 msgr2=0x7fba9c0c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.946+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 0x7fba9c0c9f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0011e0 tx=0x7fba8c00cae0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 shutdown_connections 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba8006ace0 0x7fba8006d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba9c15fca0 0x7fba9c14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba9c0b5c90 0x7fba9c0cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 --2- 172.21.15.27:0/2920950769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba9c0b6110 0x7fba9c0c9f10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:27.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 >> 172.21.15.27:0/2920950769 conn(0x7fba9c06c150 msgr2=0x7fba9c0b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:27.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 shutdown_connections 2023-07-06T20:14:27.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:27.947+0000 7fbaa41ff700 1 -- 172.21.15.27:0/2920950769 wait complete. 2023-07-06T20:14:27.966 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T20:14:27.966 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:14:27.967 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-07-06T20:14:27.967 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-06T20:14:27.967 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T20:14:27.967 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T20:14:27.967 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T20:14:27.968 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:14:27.968 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-06T20:14:27.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:27.968 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-07-06T20:14:28.015 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.016 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-07-06T20:14:28.017 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-07-06T20:14:28.020 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:14:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41942sec preferred_lft 41942sec 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 5e:d6:af:22:e7:8c brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::a0e5:ddff:fe7a:3406/64 scope link 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T20:14:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 5e:d6:af:22:e7:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T20:14:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::5cd6:afff:fe22:e78c/64 scope link 2023-07-06T20:14:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.024 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:14:28.093 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-07-06T20:14:28.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.097 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-07-06T20:14:28.146 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-07-06T20:14:28.147 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:14:28.147 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-06T20:14:28.147 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-06T20:14:28.147 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:14:28.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.148 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:28.148 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:28.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-06T20:14:28.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-06T20:14:28.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-06T20:14:28.149 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:28.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.321 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:14:28.368 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T20:14:28.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.369 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-07-06T20:14:28.415 INFO:teuthology.orchestra.run.smithi027.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-06T20:14:28.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.416 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:28.416 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:28.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T20:14:28.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-06T20:14:28.417 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:28.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T20:14:28.632 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether fa:8d:93:3a:d2:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T20:14:28.633 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T20:14:28.634 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.634 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f88d:93ff:fe3a:d221/64 scope link 2023-07-06T20:14:28.634 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.636 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T20:14:28.707 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether fa:8d:93:3a:d2:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T20:14:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T20:14:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f88d:93ff:fe3a:d221/64 scope link 2023-07-06T20:14:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:28.710 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-06T20:14:28.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.711 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:28.711 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:28.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-06T20:14:28.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T20:14:28.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-06T20:14:28.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-06T20:14:28.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-06T20:14:28.712 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:28.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:28.966 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:14:28.967 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:14:28.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.1 up 2023-07-06T20:14:28.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T20:14:28.967 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:14:29.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:29.150 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-06T20:14:29.196 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-06T20:14:29.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:29.197 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-06T20:14:29.242 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-07-06T20:14:29.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:29.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:29.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.153:3300,172.21.15.153:3301" 2023-07-06T20:14:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 2@c2a9da8e-f30b-43f7-b235-3303ae838054.cephfs=/ 2023-07-06T20:14:29.418 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.27:3300/172.21.15.153:3300/172.21.15.153:3301" will pass to kernel 2023-07-06T20:14:29.422 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:14:29.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:29.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-06T20:14:29.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:29.494 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:29.581 INFO:teuthology.orchestra.run.smithi027.stdout:client4654 2023-07-06T20:14:29.582 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:29.582 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:29.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:29.629 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:29.756 INFO:teuthology.orchestra.run.smithi027.stdout:client4654 2023-07-06T20:14:29.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:29.757 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:29.845 INFO:teuthology.orchestra.run.smithi027.stdout:client4654 2023-07-06T20:14:29.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:29.846 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2a9da8e-f30b-43f7-b235-3303ae838054.client4654/status') 2023-07-06T20:14:29.956 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.4654 (3)172.21.15.27:0/1032075290 2023-07-06T20:14:29.956 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2023-07-06T20:14:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2023-07-06T20:14:29.956 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2023-07-06T20:14:29.957 INFO:teuthology.orchestra.run.smithi027.stderr:70 bytes copied, 5.1649e-05 s, 1.4 MB/s 2023-07-06T20:14:29.959 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-07-06T20:14:29.959 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T20:14:29.959 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T20:14:29.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T20:14:30.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.160+0000 7f184abff700 1 Processor -- start 2023-07-06T20:14:30.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.160+0000 7f184abff700 1 -- start start 2023-07-06T20:14:30.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.161+0000 7f184abff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 0x7f18440d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:30.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.161+0000 7f184abff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 0x7f18440cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:30.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.161+0000 7f184abff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440cf780 con 0x7f18440b5c90 2023-07-06T20:14:30.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.161+0000 7f184abff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440cd490 con 0x7f184415fca0 2023-07-06T20:14:30.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.161+0000 7f184abff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440cd770 con 0x7f18440b6110 2023-07-06T20:14:30.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184899b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 0x7f18440d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 0x7f18440cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 0x7f18440cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:47968/0 (socket says 172.21.15.27:47968) 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184899b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 0x7f18440d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:60858/0 (socket says 172.21.15.27:60858) 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 -- 172.21.15.27:0/158334365 learned_addr learned my addr 172.21.15.27:0/158334365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 -- 172.21.15.27:0/158334365 >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 msgr2=0x7f18440d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 --2- 172.21.15.27:0/158334365 >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 0x7f18440d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 -- 172.21.15.27:0/158334365 >> v1:172.21.15.153:6789/0 conn(0x7f18440b5c90 legacy=0x7f18440d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184919c700 1 -- 172.21.15.27:0/158334365 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440cda50 con 0x7f18440b6110 2023-07-06T20:14:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.162+0000 7f184899b700 1 --2- 172.21.15.27:0/158334365 >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 0x7f18440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.163+0000 7f184919c700 1 --2- 172.21.15.27:0/158334365 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 0x7f18440cfc10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335f8c983e916594 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.163+0000 7f18437fe700 1 -- 172.21.15.27:0/158334365 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c00e070 con 0x7f18440b6110 2023-07-06T20:14:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.163+0000 7f18437fe700 1 -- 172.21.15.27:0/158334365 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c00b910 con 0x7f18440b6110 2023-07-06T20:14:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.163+0000 7f18437fe700 1 -- 172.21.15.27:0/158334365 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c00bcc0 con 0x7f18440b6110 2023-07-06T20:14:30.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.163+0000 7f184abff700 1 -- 172.21.15.27:0/158334365 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 msgr2=0x7f18440cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:30.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 --2- 172.21.15.27:0/158334365 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 0x7f18440cfc10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e50 tx=0x7f183c00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:30.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 -- 172.21.15.27:0/158334365 shutdown_connections 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 --2- 172.21.15.27:0/158334365 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18440b6110 0x7f18440cfc10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 --2- 172.21.15.27:0/158334365 >> v2:172.21.15.153:3300/0 conn(0x7f184415fca0 0x7f18440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 -- 172.21.15.27:0/158334365 >> 172.21.15.27:0/158334365 conn(0x7f184406c150 msgr2=0x7f184406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 -- 172.21.15.27:0/158334365 shutdown_connections 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.164+0000 7f184abff700 1 -- 172.21.15.27:0/158334365 wait complete. 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.165+0000 7f184abff700 1 Processor -- start 2023-07-06T20:14:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.166+0000 7f184abff700 1 -- start start 2023-07-06T20:14:30.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184abff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 0x7f18440cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:30.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184abff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18440b6110 0x7f18440c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:30.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184abff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f184415fca0 0x7f1844153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:30.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184abff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440c8730 con 0x7f18440b5c90 2023-07-06T20:14:30.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184abff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844153b40 con 0x7f18440b6110 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184abff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440cb7a0 con 0x7f184415fca0 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 0x7f18440cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 0x7f18440cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39598/0 (socket says 172.21.15.27:39598) 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 -- 172.21.15.27:0/696467111 learned_addr learned my addr 172.21.15.27:0/696467111 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184919c700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f184415fca0 0x7f1844153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f1843fff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18440b6110 0x7f18440c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 -- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f184415fca0 msgr2=0x7f1844153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f184415fca0 0x7f1844153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 -- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18440b6110 msgr2=0x7f18440c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18440b6110 0x7f18440c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.167+0000 7f184899b700 1 -- 172.21.15.27:0/696467111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380094a0 con 0x7f18440b5c90 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.168+0000 7f184899b700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 0x7f18440cc150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f183800ec10 tx=0x7f183800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.168+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838009760 con 0x7f18440b5c90 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.168+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183800c7f0 con 0x7f18440b5c90 2023-07-06T20:14:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.168+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800cb80 con 0x7f18440b5c90 2023-07-06T20:14:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.168+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c009b30 con 0x7f18440b5c90 2023-07-06T20:14:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.168+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18440c7710 con 0x7f18440b5c90 2023-07-06T20:14:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.169+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18440008d0 con 0x7f18440b5c90 2023-07-06T20:14:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.170+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f183800cce0 con 0x7f18440b5c90 2023-07-06T20:14:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.171+0000 7f18417fa700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f182806ace0 0x7f182806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.172+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1838089360 con 0x7f18440b5c90 2023-07-06T20:14:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.172+0000 7f1843fff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f182806ace0 0x7f182806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.173+0000 7f1843fff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f182806ace0 0x7f182806d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18440cbbf0 tx=0x7f1830009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:30.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.175+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f18380544a0 con 0x7f18440b5c90 2023-07-06T20:14:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.429+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18440d06b0 con 0x7f18440b5c90 2023-07-06T20:14:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.430+0000 7f18417fa700 1 -- 172.21.15.27:0/696467111 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1838021060 con 0x7f18440b5c90 2023-07-06T20:14:30.433 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:14:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f182806ace0 msgr2=0x7f182806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f182806ace0 0x7f182806d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18440cbbf0 tx=0x7f1830009040 comp rx=0 tx=0).stop 2023-07-06T20:14:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 msgr2=0x7f18440cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 0x7f18440cc150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f183800ec10 tx=0x7f183800ef20 comp rx=0 tx=0).stop 2023-07-06T20:14:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 shutdown_connections 2023-07-06T20:14:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f182806ace0 0x7f182806d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f184415fca0 0x7f1844153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18440b5c90 0x7f18440cc150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 --2- 172.21.15.27:0/696467111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18440b6110 0x7f18440c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 >> 172.21.15.27:0/696467111 conn(0x7f184406c150 msgr2=0x7f18440b81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 shutdown_connections 2023-07-06T20:14:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:30.432+0000 7f184abff700 1 -- 172.21.15.27:0/696467111 wait complete. 2023-07-06T20:14:30.452 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T20:14:30.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:14:30.453 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-07-06T20:14:30.453 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T20:14:30.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T20:14:30.453 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T20:14:30.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T20:14:30.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:14:30.454 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T20:14:30.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:30.455 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-07-06T20:14:30.506 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:30.506 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:30.506 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:30.506 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:30.506 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:30.506 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-07-06T20:14:30.507 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 41831sec preferred_lft 41831sec 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2023-07-06T20:14:30.508 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:30.509 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T20:14:30.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:30.510 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:30.510 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:30.510 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add name ceph-brx type bridge 2023-07-06T20:14:30.510 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr flush dev ceph-brx 2023-07-06T20:14:30.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set ceph-brx up 2023-07-06T20:14:30.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T20:14:30.511 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:30.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:30.996 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:14:31.099 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-07-06T20:14:31.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:31.102 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-07-06T20:14:31.174 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-07-06T20:14:31.174 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:14:31.174 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T20:14:31.174 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T20:14:31.174 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:14:31.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:31.176 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:31.176 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:31.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T20:14:31.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T20:14:31.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T20:14:31.177 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:31.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:31.856 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:14:31.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:31.903 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2023-07-06T20:14:31.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:31.950 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:31.951 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:31.951 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T20:14:31.951 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-06T20:14:31.951 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:32.096 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-06T20:14:32.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:32.097 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:32.097 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:32.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-06T20:14:32.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T20:14:32.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-06T20:14:32.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-06T20:14:32.099 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-06T20:14:32.099 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:32.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:32.360 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:32.360 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:32.360 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-07-06T20:14:32.360 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T20:14:32.361 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:32.538 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:32.539 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T20:14:32.584 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T20:14:32.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:32.584 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T20:14:32.669 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-07-06T20:14:32.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:32.669 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:32.816 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:32.817 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:33.255 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:33.256 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:33.256 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.153:3300,172.21.15.153:3301" 2023-07-06T20:14:33.256 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with new device syntax: 1@c2a9da8e-f30b-43f7-b235-3303ae838054.cephfs=/ 2023-07-06T20:14:33.256 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.153:3300/172.21.15.153:3301" will pass to kernel 2023-07-06T20:14:33.261 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:14:33.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:33.262 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T20:14:33.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:33.332 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:33.428 INFO:teuthology.orchestra.run.smithi153.stdout:client4713 2023-07-06T20:14:33.428 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:33.429 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:33.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:33.474 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:33.602 INFO:teuthology.orchestra.run.smithi153.stdout:client4713 2023-07-06T20:14:33.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:33.602 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:33.689 INFO:teuthology.orchestra.run.smithi153.stdout:client4713 2023-07-06T20:14:33.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:33.690 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2a9da8e-f30b-43f7-b235-3303ae838054.client4713/status') 2023-07-06T20:14:33.797 INFO:teuthology.orchestra.run.smithi153.stdout:instance: client.4713 (3)172.21.15.153:0/3610769461 2023-07-06T20:14:33.797 INFO:teuthology.orchestra.run.smithi153.stdout:blocklisted: false 2023-07-06T20:14:33.797 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records in 2023-07-06T20:14:33.798 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records out 2023-07-06T20:14:33.798 INFO:teuthology.orchestra.run.smithi153.stderr:71 bytes copied, 5.1406e-05 s, 1.4 MB/s 2023-07-06T20:14:33.799 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-07-06T20:14:33.800 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T20:14:33.800 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T20:14:33.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T20:14:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.013+0000 7f13e51ff700 1 Processor -- start 2023-07-06T20:14:34.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.014+0000 7f13e51ff700 1 -- start start 2023-07-06T20:14:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13e51ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13e0164530 0x7f13e006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13e51ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e00b0440 0x7f13e00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13e51ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e00cc3b0 con 0x7f13e0160900 2023-07-06T20:14:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13e51ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e00b0e00 con 0x7f13e0164530 2023-07-06T20:14:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13e51ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e01649f0 con 0x7f13e00b0440 2023-07-06T20:14:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13d7fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13e0164530 0x7f13e006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:34.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13d7fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13e0164530 0x7f13e006b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45016/0 (socket says 172.21.15.27:45016) 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13d7fff700 1 -- 172.21.15.27:0/3997065801 learned_addr learned my addr 172.21.15.27:0/3997065801 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.015+0000 7f13df59e700 1 --2- 172.21.15.27:0/3997065801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e00b0440 0x7f13e00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13d7fff700 1 -- 172.21.15.27:0/3997065801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e00b0440 msgr2=0x7f13e00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13d7fff700 1 --2- 172.21.15.27:0/3997065801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e00b0440 0x7f13e00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13d7fff700 1 -- 172.21.15.27:0/3997065801 >> v1:172.21.15.153:6789/0 conn(0x7f13e0160900 legacy=0x7f13e00adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13d7fff700 1 -- 172.21.15.27:0/3997065801 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0164cd0 con 0x7f13e0164530 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13d7fff700 1 --2- 172.21.15.27:0/3997065801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e006b220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13c80098f0 tx=0x7f13c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d162fa9bacdd6ab7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:34.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13de59c700 1 -- 172.21.15.27:0/3997065801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8011830 con 0x7f13e0164530 2023-07-06T20:14:34.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13de59c700 1 -- 172.21.15.27:0/3997065801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8011990 con 0x7f13e0164530 2023-07-06T20:14:34.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.016+0000 7f13de59c700 1 -- 172.21.15.27:0/3997065801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8011d20 con 0x7f13e0164530 2023-07-06T20:14:34.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 -- 172.21.15.27:0/3997065801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 msgr2=0x7f13e006b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:34.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 --2- 172.21.15.27:0/3997065801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e006b220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13c80098f0 tx=0x7f13c8009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 -- 172.21.15.27:0/3997065801 shutdown_connections 2023-07-06T20:14:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 --2- 172.21.15.27:0/3997065801 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e00b0440 0x7f13e00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 --2- 172.21.15.27:0/3997065801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e006b220 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 -- 172.21.15.27:0/3997065801 >> 172.21.15.27:0/3997065801 conn(0x7f13e00cb440 msgr2=0x7f13e00cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:34.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 -- 172.21.15.27:0/3997065801 shutdown_connections 2023-07-06T20:14:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.017+0000 7f13e51ff700 1 -- 172.21.15.27:0/3997065801 wait complete. 2023-07-06T20:14:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.019+0000 7f13e51ff700 1 Processor -- start 2023-07-06T20:14:34.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.019+0000 7f13e51ff700 1 -- start start 2023-07-06T20:14:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13e51ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13e00b0440 0x7f13e015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:34.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13e51ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e0160900 0x7f13e00c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13e51ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e00c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13e51ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0161e10 con 0x7f13e0164530 2023-07-06T20:14:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13e51ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0160d70 con 0x7f13e00b0440 2023-07-06T20:14:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13e51ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0161050 con 0x7f13e0160900 2023-07-06T20:14:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13df59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e00c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13df59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e00c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45032/0 (socket says 172.21.15.27:45032) 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13d7fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e0160900 0x7f13e00c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13df59e700 1 -- 172.21.15.27:0/284622333 learned_addr learned my addr 172.21.15.27:0/284622333 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.020+0000 7f13ded9d700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13e00b0440 0x7f13e015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d7fff700 1 -- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13e00b0440 msgr2=0x7f13e015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d7fff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13e00b0440 0x7f13e015f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d7fff700 1 -- 172.21.15.27:0/284622333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 msgr2=0x7f13e00c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d7fff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e00c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d7fff700 1 -- 172.21.15.27:0/284622333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8009550 con 0x7f13e0160900 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13df59e700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e00c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d7fff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e0160900 0x7f13e00c7c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13c8016000 tx=0x7f13c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8004400 con 0x7f13e0160900 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c801f070 con 0x7f13e0160900 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e0161330 con 0x7f13e0160900 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.021+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80038e0 con 0x7f13e0160900 2023-07-06T20:14:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.022+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e01618f0 con 0x7f13e0160900 2023-07-06T20:14:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.022+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e00008d0 con 0x7f13e0160900 2023-07-06T20:14:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.024+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13c8021030 con 0x7f13e0160900 2023-07-06T20:14:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.024+0000 7f13d77fe700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f13c006ace0 0x7f13c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.025+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13c808a790 con 0x7f13e0160900 2023-07-06T20:14:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.025+0000 7f13ded9d700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f13c006ace0 0x7f13c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.025+0000 7f13ded9d700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f13c006ace0 0x7f13c006d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13d0007fd0 tx=0x7f13d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.030+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13c8055850 con 0x7f13e0160900 2023-07-06T20:14:34.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.280+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f13e00b4940 con 0x7f13e0160900 2023-07-06T20:14:34.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.280+0000 7f13d77fe700 1 -- 172.21.15.27:0/284622333 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f13c8058e70 con 0x7f13e0160900 2023-07-06T20:14:34.283 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f13c006ace0 msgr2=0x7f13c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f13c006ace0 0x7f13c006d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13d0007fd0 tx=0x7f13d0009040 comp rx=0 tx=0).stop 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e0160900 msgr2=0x7f13e00c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e0160900 0x7f13e00c7c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13c8016000 tx=0x7f13c8004750 comp rx=0 tx=0).stop 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 shutdown_connections 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f13c006ace0 0x7f13c006d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e0164530 0x7f13e00c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13e00b0440 0x7f13e015f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 --2- 172.21.15.27:0/284622333 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13e0160900 0x7f13e00c7c60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:34.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 >> 172.21.15.27:0/284622333 conn(0x7f13e00cb440 msgr2=0x7f13e00aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:34.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 shutdown_connections 2023-07-06T20:14:34.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:34.283+0000 7f13e51ff700 1 -- 172.21.15.27:0/284622333 wait complete. 2023-07-06T20:14:34.302 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T20:14:34.302 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:14:34.303 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-07-06T20:14:34.303 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-06T20:14:34.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T20:14:34.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T20:14:34.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T20:14:34.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:14:34.304 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-06T20:14:34.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.304 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-07-06T20:14:34.351 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:34.351 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:34.351 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:34.351 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.351 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:34.352 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.352 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:34.352 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:34.352 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-07-06T20:14:34.352 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T20:14:34.353 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 41827sec preferred_lft 41827sec 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether d2:d2:ab:a1:94:c6 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::90a5:cff:fe20:b613/64 scope link 2023-07-06T20:14:34.354 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.355 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T20:14:34.355 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether d2:d2:ab:a1:94:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T20:14:34.355 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::d0d2:abff:fea1:94c6/64 scope link tentative 2023-07-06T20:14:34.355 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.356 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:14:34.423 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-07-06T20:14:34.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.426 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-07-06T20:14:34.474 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-07-06T20:14:34.475 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:14:34.475 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T20:14:34.475 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T20:14:34.475 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:14:34.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.476 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:34.477 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:34.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T20:14:34.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T20:14:34.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T20:14:34.478 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:34.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.646 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:14:34.691 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T20:14:34.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.693 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2023-07-06T20:14:34.738 INFO:teuthology.orchestra.run.smithi153.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-06T20:14:34.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.740 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:34.740 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:34.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T20:14:34.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-06T20:14:34.741 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:34.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T20:14:34.991 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:34.992 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:34.992 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:34.992 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.992 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:34.992 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.992 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:14:34.993 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 6a:e7:9d:ed:20:7b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T20:14:34.993 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T20:14:34.993 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.993 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::68e7:9dff:feed:207b/64 scope link 2023-07-06T20:14:34.993 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:34.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:34.995 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T20:14:35.064 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:14:35.065 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:14:35.065 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:14:35.065 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:35.065 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-07-06T20:14:35.065 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:35.065 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:14:35.066 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 6a:e7:9d:ed:20:7b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T20:14:35.066 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T20:14:35.066 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:35.066 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::68e7:9dff:feed:207b/64 scope link 2023-07-06T20:14:35.066 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:14:35.067 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-06T20:14:35.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:35.068 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:35.068 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:35.069 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-06T20:14:35.069 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T20:14:35.070 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-06T20:14:35.070 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-06T20:14:35.070 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-06T20:14:35.071 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:35.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:35.310 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:14:35.310 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:14:35.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.1 up 2023-07-06T20:14:35.311 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T20:14:35.311 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:14:35.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:35.490 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-06T20:14:35.535 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-06T20:14:35.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:14:35.535 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-06T20:14:35.619 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-07-06T20:14:35.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:35.620 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:35.763 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:35.763 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:35.791 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:14:35.791 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:14:35.791 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.153:3300,172.21.15.153:3301" 2023-07-06T20:14:35.792 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with new device syntax: 3@c2a9da8e-f30b-43f7-b235-3303ae838054.cephfs=/ 2023-07-06T20:14:35.792 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.27:3300/172.21.15.153:3300/172.21.15.153:3301" will pass to kernel 2023-07-06T20:14:35.795 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:14:35.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:35.796 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-06T20:14:35.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:35.865 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:35.953 INFO:teuthology.orchestra.run.smithi153.stdout:client4722 2023-07-06T20:14:35.954 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:35.955 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:36.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:36.001 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:36.127 INFO:teuthology.orchestra.run.smithi153.stdout:client4722 2023-07-06T20:14:36.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:36.128 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:14:36.215 INFO:teuthology.orchestra.run.smithi153.stdout:client4722 2023-07-06T20:14:36.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:36.216 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c2a9da8e-f30b-43f7-b235-3303ae838054.client4722/status') 2023-07-06T20:14:36.322 INFO:teuthology.orchestra.run.smithi153.stdout:instance: client.4722 (3)172.21.15.153:0/1917560969 2023-07-06T20:14:36.323 INFO:teuthology.orchestra.run.smithi153.stdout:blocklisted: false 2023-07-06T20:14:36.323 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records in 2023-07-06T20:14:36.323 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records out 2023-07-06T20:14:36.323 INFO:teuthology.orchestra.run.smithi153.stderr:71 bytes copied, 5.3839e-05 s, 1.3 MB/s 2023-07-06T20:14:36.325 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-06T20:14:36.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:36.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.555+0000 7fab8e1ff700 1 Processor -- start 2023-07-06T20:14:36.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.556+0000 7fab8e1ff700 1 -- start start 2023-07-06T20:14:36.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab8e1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fab880b5c90 0x7fab880d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab8e1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 0x7fab880d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab8e1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab880cf780 con 0x7fab8815fca0 2023-07-06T20:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab8e1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab880cd490 con 0x7fab880b5c90 2023-07-06T20:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab8e1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880cd770 con 0x7fab880b6110 2023-07-06T20:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab86ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fab880b5c90 0x7fab880d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab87fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 0x7fab880d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab87fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 0x7fab880d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:34254/0 (socket says 172.21.15.27:34254) 2023-07-06T20:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab877fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fab8815fca0 0x7fab880d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:41890/0 (socket says 172.21.15.27:41890) 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.557+0000 7fab87fff700 1 -- 172.21.15.27:0/700745764 learned_addr learned my addr 172.21.15.27:0/700745764 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab87fff700 1 -- 172.21.15.27:0/700745764 >> v2:172.21.15.153:3300/0 conn(0x7fab880b5c90 msgr2=0x7fab880d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab87fff700 1 --2- 172.21.15.27:0/700745764 >> v2:172.21.15.153:3300/0 conn(0x7fab880b5c90 0x7fab880d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab867fc700 1 -- 172.21.15.27:0/700745764 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946561933 0 0) 0x7fab880cf780 con 0x7fab8815fca0 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab87fff700 1 -- 172.21.15.27:0/700745764 >> v1:172.21.15.153:6789/0 conn(0x7fab8815fca0 legacy=0x7fab880d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab87fff700 1 -- 172.21.15.27:0/700745764 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab880cda50 con 0x7fab880b6110 2023-07-06T20:14:36.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab87fff700 1 --2- 172.21.15.27:0/700745764 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 0x7fab880d3280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab80009e50 tx=0x7fab8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409f8c5052ecc802 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:36.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab867fc700 1 -- 172.21.15.27:0/700745764 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000e070 con 0x7fab880b6110 2023-07-06T20:14:36.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab867fc700 1 -- 172.21.15.27:0/700745764 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8000b910 con 0x7fab880b6110 2023-07-06T20:14:36.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.558+0000 7fab867fc700 1 -- 172.21.15.27:0/700745764 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000bcc0 con 0x7fab880b6110 2023-07-06T20:14:36.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 -- 172.21.15.27:0/700745764 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 msgr2=0x7fab880d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:36.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/700745764 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 0x7fab880d3280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab80009e50 tx=0x7fab8000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:36.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 -- 172.21.15.27:0/700745764 shutdown_connections 2023-07-06T20:14:36.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/700745764 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab880b6110 0x7fab880d3280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/700745764 >> v2:172.21.15.153:3300/0 conn(0x7fab880b5c90 0x7fab880d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 -- 172.21.15.27:0/700745764 >> 172.21.15.27:0/700745764 conn(0x7fab8806c150 msgr2=0x7fab8806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:36.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 -- 172.21.15.27:0/700745764 shutdown_connections 2023-07-06T20:14:36.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.559+0000 7fab8e1ff700 1 -- 172.21.15.27:0/700745764 wait complete. 2023-07-06T20:14:36.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.561+0000 7fab8e1ff700 1 Processor -- start 2023-07-06T20:14:36.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.561+0000 7fab8e1ff700 1 -- start start 2023-07-06T20:14:36.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab8e1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 0x7fab880d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:36.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab8e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab880b6110 0x7fab880d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab8e1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab8815fca0 0x7fab88159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab8e1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b5860 con 0x7fab880b5c90 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab8e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880c97f0 con 0x7fab880b6110 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab8e1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880c6930 con 0x7fab8815fca0 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 0x7fab880d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.562+0000 7fab877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 0x7fab880d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45036/0 (socket says 172.21.15.27:45036) 2023-07-06T20:14:36.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 -- 172.21.15.27:0/2685198056 learned_addr learned my addr 172.21.15.27:0/2685198056 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab86ffd700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab880b6110 0x7fab880d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab87fff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab8815fca0 0x7fab88159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 -- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab8815fca0 msgr2=0x7fab88159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab8815fca0 0x7fab88159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 -- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab880b6110 msgr2=0x7fab880d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab880b6110 0x7fab880d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 -- 172.21.15.27:0/2685198056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab80009b30 con 0x7fab880b5c90 2023-07-06T20:14:36.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab877fe700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 0x7fab880d0960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab7800c550 tx=0x7fab7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:36.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.563+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab78004c80 con 0x7fab880b5c90 2023-07-06T20:14:36.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.564+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab78004e00 con 0x7fab880b5c90 2023-07-06T20:14:36.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.564+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab78007310 con 0x7fab880b5c90 2023-07-06T20:14:36.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.564+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab880b2a60 con 0x7fab880b5c90 2023-07-06T20:14:36.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.564+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab880c6e60 con 0x7fab880b5c90 2023-07-06T20:14:36.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.565+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab880b2ca0 con 0x7fab880b5c90 2023-07-06T20:14:36.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.566+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab78007470 con 0x7fab880b5c90 2023-07-06T20:14:36.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.567+0000 7fab6ffff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fab7406ace0 0x7fab7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:36.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.567+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab78088ff0 con 0x7fab880b5c90 2023-07-06T20:14:36.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.567+0000 7fab86ffd700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fab7406ace0 0x7fab7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:36.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.568+0000 7fab86ffd700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fab7406ace0 0x7fab7406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009be0 tx=0x7fab7c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:36.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.571+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab880b2ca0 con 0x7fab880b5c90 2023-07-06T20:14:36.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.783+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab880b2ca0 con 0x7fab880b5c90 2023-07-06T20:14:36.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.784+0000 7fab6ffff700 1 -- 172.21.15.27:0/2685198056 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18255 (secure 0 0 0) 0x7fab880b2ca0 con 0x7fab880b5c90 2023-07-06T20:14:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:36.787 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:01.036977+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fab7406ace0 msgr2=0x7fab7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fab7406ace0 0x7fab7406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009be0 tx=0x7fab7c009210 comp rx=0 tx=0).stop 2023-07-06T20:14:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 msgr2=0x7fab880d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 0x7fab880d0960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab7800c550 tx=0x7fab7800c860 comp rx=0 tx=0).stop 2023-07-06T20:14:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 shutdown_connections 2023-07-06T20:14:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fab7406ace0 0x7fab7406d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab8815fca0 0x7fab88159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab880b5c90 0x7fab880d0960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 --2- 172.21.15.27:0/2685198056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab880b6110 0x7fab880d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 >> 172.21.15.27:0/2685198056 conn(0x7fab8806c150 msgr2=0x7fab8815eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 shutdown_connections 2023-07-06T20:14:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:36.787+0000 7fab8e1ff700 1 -- 172.21.15.27:0/2685198056 wait complete. 2023-07-06T20:14:36.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:36.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.018+0000 7f6768bff700 1 Processor -- start 2023-07-06T20:14:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.018+0000 7f6768bff700 1 -- start start 2023-07-06T20:14:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.019+0000 7f6768bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67640cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.019+0000 7f6768bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67640cf780 con 0x7f676415fca0 2023-07-06T20:14:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.019+0000 7f6768bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67640cd490 con 0x7f67640b5c90 2023-07-06T20:14:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.019+0000 7f6768bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640cd770 con 0x7f67640b6110 2023-07-06T20:14:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.019+0000 7f676259c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f676415fca0 0x7f67640d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60992/0 (socket says 172.21.15.27:60992) 2023-07-06T20:14:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.019+0000 7f676259c700 1 -- 172.21.15.27:0/812047698 learned_addr learned my addr 172.21.15.27:0/812047698 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6762d9d700 1 --2- 172.21.15.27:0/812047698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67640cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938460170 0 0) 0x7f67640cf780 con 0x7f676415fca0 2023-07-06T20:14:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67500035b0 con 0x7f676415fca0 2023-07-06T20:14:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583653243 0 0) 0x7f67640cd490 con 0x7f67640b5c90 2023-07-06T20:14:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67640cf780 con 0x7f67640b5c90 2023-07-06T20:14:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6762d9d700 1 -- 172.21.15.27:0/812047698 >> v1:172.21.15.153:6789/0 conn(0x7f67640b5c90 legacy=0x7f67640d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6762d9d700 1 -- 172.21.15.27:0/812047698 >> v1:172.21.15.27:6789/0 conn(0x7f676415fca0 legacy=0x7f67640d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6762d9d700 1 -- 172.21.15.27:0/812047698 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640cda50 con 0x7f67640b6110 2023-07-06T20:14:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.020+0000 7f6762d9d700 1 --2- 172.21.15.27:0/812047698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67640cfc10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f675c009dc0 tx=0x7f675c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351dfaafb1f340a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00e070 con 0x7f67640b6110 2023-07-06T20:14:37.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c00bb20 con 0x7f67640b6110 2023-07-06T20:14:37.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6761d9b700 1 -- 172.21.15.27:0/812047698 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c012430 con 0x7f67640b6110 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 -- 172.21.15.27:0/812047698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 msgr2=0x7f67640cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 --2- 172.21.15.27:0/812047698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67640cfc10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f675c009dc0 tx=0x7f675c00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 -- 172.21.15.27:0/812047698 shutdown_connections 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 --2- 172.21.15.27:0/812047698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67640cfc10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 -- 172.21.15.27:0/812047698 >> 172.21.15.27:0/812047698 conn(0x7f676406c150 msgr2=0x7f676406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 -- 172.21.15.27:0/812047698 shutdown_connections 2023-07-06T20:14:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.021+0000 7f6768bff700 1 -- 172.21.15.27:0/812047698 wait complete. 2023-07-06T20:14:37.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.024+0000 7f6768bff700 1 Processor -- start 2023-07-06T20:14:37.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.024+0000 7f6768bff700 1 -- start start 2023-07-06T20:14:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.025+0000 7f6768bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 0x7f67640cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.025+0000 7f6768bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67641f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.025+0000 7f6768bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f676415fca0 0x7f67641f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.025+0000 7f6768bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640ccc40 con 0x7f67640b5c90 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.025+0000 7f6768bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6764157bd0 con 0x7f676415fca0 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.025+0000 7f6768bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676415c3a0 con 0x7f67640b6110 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 0x7f67640cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 0x7f67640cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45050/0 (socket says 172.21.15.27:45050) 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 -- 172.21.15.27:0/2043101144 learned_addr learned my addr 172.21.15.27:0/2043101144 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f675bfff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67641f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f6762d9d700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f676415fca0 0x7f67641f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 -- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 msgr2=0x7f67641f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67641f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 -- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f676415fca0 msgr2=0x7f67641f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f676415fca0 0x7f67641f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 -- 172.21.15.27:0/2043101144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c009a70 con 0x7f67640b5c90 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f675bfff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67641f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.026+0000 7f676259c700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 0x7f67640cc700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f675400c4d0 tx=0x7f675400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.027+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754005d70 con 0x7f67640b5c90 2023-07-06T20:14:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.027+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67540070d0 con 0x7f67640b5c90 2023-07-06T20:14:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.027+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676415d590 con 0x7f67640b5c90 2023-07-06T20:14:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.027+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754007460 con 0x7f67640b5c90 2023-07-06T20:14:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.027+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67641f5430 con 0x7f67640b5c90 2023-07-06T20:14:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.028+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67640008d0 con 0x7f67640b5c90 2023-07-06T20:14:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.029+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67540075c0 con 0x7f67640b5c90 2023-07-06T20:14:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.030+0000 7f675affd700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f674406ace0 0x7f674406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.030+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67540888f0 con 0x7f67640b5c90 2023-07-06T20:14:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.030+0000 7f675bfff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f674406ace0 0x7f674406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.031+0000 7f675bfff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f674406ace0 0x7f674406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f674c007fd0 tx=0x7f674c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.034+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6754053990 con 0x7f67640b5c90 2023-07-06T20:14:37.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.239+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67640d2330 con 0x7f67640b5c90 2023-07-06T20:14:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.240+0000 7f675affd700 1 -- 172.21.15.27:0/2043101144 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6754056fb0 con 0x7f67640b5c90 2023-07-06T20:14:37.242 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f674406ace0 msgr2=0x7f674406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f674406ace0 0x7f674406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f674c007fd0 tx=0x7f674c009040 comp rx=0 tx=0).stop 2023-07-06T20:14:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 msgr2=0x7f67640cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 0x7f67640cc700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f675400c4d0 tx=0x7f675400c7e0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 shutdown_connections 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f674406ace0 0x7f674406d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f676415fca0 0x7f67641f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67640b5c90 0x7f67640cc700 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 --2- 172.21.15.27:0/2043101144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67640b6110 0x7f67641f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 >> 172.21.15.27:0/2043101144 conn(0x7f676406c150 msgr2=0x7f67640b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 shutdown_connections 2023-07-06T20:14:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.242+0000 7f6768bff700 1 -- 172.21.15.27:0/2043101144 wait complete. 2023-07-06T20:14:37.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:37.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.468+0000 7f1abadff700 1 Processor -- start 2023-07-06T20:14:37.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.468+0000 7f1abadff700 1 -- start start 2023-07-06T20:14:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1abadff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ab415fca0 0x7f1ab40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1abadff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1ab40b5c90 0x7f1ab40d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1abadff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40cf780 con 0x7f1ab40b6110 2023-07-06T20:14:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1abadff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab40cd490 con 0x7f1ab415fca0 2023-07-06T20:14:37.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1abadff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab40cd770 con 0x7f1ab40b5c90 2023-07-06T20:14:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1ab939c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1ab40b6110 0x7f1ab40cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:32776/0 (socket says 172.21.15.27:32776) 2023-07-06T20:14:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1ab8b9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ab415fca0 0x7f1ab40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.469+0000 7f1ab939c700 1 -- 172.21.15.27:0/4281057880 learned_addr learned my addr 172.21.15.27:0/4281057880 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:37.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab3fff700 1 --2- 172.21.15.27:0/4281057880 >> v2:172.21.15.153:3300/0 conn(0x7f1ab40b5c90 0x7f1ab40d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab8b9b700 1 -- 172.21.15.27:0/4281057880 >> v2:172.21.15.153:3300/0 conn(0x7f1ab40b5c90 msgr2=0x7f1ab40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab8b9b700 1 --2- 172.21.15.27:0/4281057880 >> v2:172.21.15.153:3300/0 conn(0x7f1ab40b5c90 0x7f1ab40d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab8b9b700 1 -- 172.21.15.27:0/4281057880 >> v1:172.21.15.27:6789/0 conn(0x7f1ab40b6110 legacy=0x7f1ab40cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab8b9b700 1 -- 172.21.15.27:0/4281057880 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40cda50 con 0x7f1ab415fca0 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab8b9b700 1 --2- 172.21.15.27:0/4281057880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab415fca0 0x7f1ab40d1420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406b6f0 tx=0x7f1aa8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9389dbf832d220 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.470+0000 7f1ab37fe700 1 -- 172.21.15.27:0/4281057880 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8011830 con 0x7f1ab415fca0 2023-07-06T20:14:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1ab37fe700 1 -- 172.21.15.27:0/4281057880 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa8011990 con 0x7f1ab415fca0 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1ab37fe700 1 -- 172.21.15.27:0/4281057880 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8011d20 con 0x7f1ab415fca0 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 -- 172.21.15.27:0/4281057880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab415fca0 msgr2=0x7f1ab40d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 --2- 172.21.15.27:0/4281057880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab415fca0 0x7f1ab40d1420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406b6f0 tx=0x7f1aa8009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 -- 172.21.15.27:0/4281057880 shutdown_connections 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 --2- 172.21.15.27:0/4281057880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab415fca0 0x7f1ab40d1420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 --2- 172.21.15.27:0/4281057880 >> v2:172.21.15.153:3300/0 conn(0x7f1ab40b5c90 0x7f1ab40d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 -- 172.21.15.27:0/4281057880 >> 172.21.15.27:0/4281057880 conn(0x7f1ab406c150 msgr2=0x7f1ab406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:37.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 -- 172.21.15.27:0/4281057880 shutdown_connections 2023-07-06T20:14:37.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.471+0000 7f1abadff700 1 -- 172.21.15.27:0/4281057880 wait complete. 2023-07-06T20:14:37.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.473+0000 7f1abadff700 1 Processor -- start 2023-07-06T20:14:37.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.473+0000 7f1abadff700 1 -- start start 2023-07-06T20:14:37.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.474+0000 7f1abadff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab40b5c90 0x7f1ab415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.474+0000 7f1abadff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 0x7f1ab40c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.474+0000 7f1abadff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab415fca0 0x7f1ab40c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.474+0000 7f1abadff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab41f20e0 con 0x7f1ab40b6110 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.474+0000 7f1abadff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab41f65d0 con 0x7f1ab415fca0 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.474+0000 7f1abadff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415f1e0 con 0x7f1ab40b5c90 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 0x7f1ab40c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 0x7f1ab40c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45066/0 (socket says 172.21.15.27:45066) 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 -- 172.21.15.27:0/3095226926 learned_addr learned my addr 172.21.15.27:0/3095226926 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab939c700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab415fca0 0x7f1ab40c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab8b9b700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab40b5c90 0x7f1ab415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 -- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab40b5c90 msgr2=0x7f1ab415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab40b5c90 0x7f1ab415eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 -- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab415fca0 msgr2=0x7f1ab40c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab415fca0 0x7f1ab40c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 -- 172.21.15.27:0/3095226926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80094a0 con 0x7f1ab40b6110 2023-07-06T20:14:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.475+0000 7f1ab3fff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 0x7f1ab40c6910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000e9b0 tx=0x7f1aa000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.476+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0009710 con 0x7f1ab40b6110 2023-07-06T20:14:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.476+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa000ee50 con 0x7f1ab40b6110 2023-07-06T20:14:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.476+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab41f5a50 con 0x7f1ab40b6110 2023-07-06T20:14:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.476+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa000c9f0 con 0x7f1ab40b6110 2023-07-06T20:14:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.476+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab41f6010 con 0x7f1ab40b6110 2023-07-06T20:14:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.477+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab40008d0 con 0x7f1ab40b6110 2023-07-06T20:14:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.478+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1aa000cb50 con 0x7f1ab40b6110 2023-07-06T20:14:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.479+0000 7f1ab17fa700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1a9806ace0 0x7f1a9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.479+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1aa0089530 con 0x7f1ab40b6110 2023-07-06T20:14:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.480+0000 7f1ab8b9b700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1a9806ace0 0x7f1a9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.480+0000 7f1ab8b9b700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1a9806ace0 0x7f1a9806d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80090b0 tx=0x7f1aa801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.483+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1aa0057fd0 con 0x7f1ab40b6110 2023-07-06T20:14:37.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.689+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ab40b2df0 con 0x7f1ab40b6110 2023-07-06T20:14:37.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.689+0000 7f1ab17fa700 1 -- 172.21.15.27:0/3095226926 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1aa0057b60 con 0x7f1ab40b6110 2023-07-06T20:14:37.691 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:37.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1a9806ace0 msgr2=0x7f1a9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1a9806ace0 0x7f1a9806d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80090b0 tx=0x7f1aa801c040 comp rx=0 tx=0).stop 2023-07-06T20:14:37.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 msgr2=0x7f1ab40c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 0x7f1ab40c6910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000e9b0 tx=0x7f1aa000ecc0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 shutdown_connections 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1a9806ace0 0x7f1a9806d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab415fca0 0x7f1ab40c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab40b5c90 0x7f1ab415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 --2- 172.21.15.27:0/3095226926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40b6110 0x7f1ab40c6910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 >> 172.21.15.27:0/3095226926 conn(0x7f1ab406c150 msgr2=0x7f1ab40b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.691+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 shutdown_connections 2023-07-06T20:14:37.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.692+0000 7f1abadff700 1 -- 172.21.15.27:0/3095226926 wait complete. 2023-07-06T20:14:37.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:37.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.916+0000 7f12b9fff700 1 Processor -- start 2023-07-06T20:14:37.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.916+0000 7f12b9fff700 1 -- start start 2023-07-06T20:14:37.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.917+0000 7f12b9fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12b40b5c90 0x7f12b40d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b9fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12b40b6110 0x7f12b40d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b9fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40cf780 con 0x7f12b415fca0 2023-07-06T20:14:37.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b9fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b40cd490 con 0x7f12b40b5c90 2023-07-06T20:14:37.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b9fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b40cd770 con 0x7f12b40b6110 2023-07-06T20:14:37.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b2ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12b40b5c90 0x7f12b40d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b37fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f12b415fca0 0x7f12b40d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:41916/0 (socket says 172.21.15.27:41916) 2023-07-06T20:14:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b37fe700 1 -- 172.21.15.27:0/3746311924 learned_addr learned my addr 172.21.15.27:0/3746311924 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b3fff700 1 --2- 172.21.15.27:0/3746311924 >> v2:172.21.15.153:3300/0 conn(0x7f12b40b6110 0x7f12b40d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b2ffd700 1 -- 172.21.15.27:0/3746311924 >> v2:172.21.15.153:3300/0 conn(0x7f12b40b6110 msgr2=0x7f12b40d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b2ffd700 1 --2- 172.21.15.27:0/3746311924 >> v2:172.21.15.153:3300/0 conn(0x7f12b40b6110 0x7f12b40d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b2ffd700 1 -- 172.21.15.27:0/3746311924 >> v1:172.21.15.153:6789/0 conn(0x7f12b415fca0 legacy=0x7f12b40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b2ffd700 1 -- 172.21.15.27:0/3746311924 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b40cda50 con 0x7f12b40b5c90 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.918+0000 7f12b27fc700 1 -- 172.21.15.27:0/3746311924 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030826755 0 0) 0x7f12b40cf780 con 0x7f12b415fca0 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.919+0000 7f12b2ffd700 1 --2- 172.21.15.27:0/3746311924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b40b5c90 0x7f12b40d2d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12b40cd410 tx=0x7f12a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77478e3a9878cf01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.919+0000 7f12b27fc700 1 -- 172.21.15.27:0/3746311924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011830 con 0x7f12b40b5c90 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.919+0000 7f12b27fc700 1 -- 172.21.15.27:0/3746311924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8011990 con 0x7f12b40b5c90 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.919+0000 7f12b27fc700 1 -- 172.21.15.27:0/3746311924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011d20 con 0x7f12b40b5c90 2023-07-06T20:14:37.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.919+0000 7f12b9fff700 1 -- 172.21.15.27:0/3746311924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b40b5c90 msgr2=0x7f12b40d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.919+0000 7f12b9fff700 1 --2- 172.21.15.27:0/3746311924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b40b5c90 0x7f12b40d2d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12b40cd410 tx=0x7f12a8009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.920+0000 7f12b9fff700 1 -- 172.21.15.27:0/3746311924 shutdown_connections 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.920+0000 7f12b9fff700 1 --2- 172.21.15.27:0/3746311924 >> v2:172.21.15.153:3300/0 conn(0x7f12b40b6110 0x7f12b40d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.920+0000 7f12b9fff700 1 --2- 172.21.15.27:0/3746311924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b40b5c90 0x7f12b40d2d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.920+0000 7f12b9fff700 1 -- 172.21.15.27:0/3746311924 >> 172.21.15.27:0/3746311924 conn(0x7f12b406c150 msgr2=0x7f12b406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.920+0000 7f12b9fff700 1 -- 172.21.15.27:0/3746311924 shutdown_connections 2023-07-06T20:14:37.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.920+0000 7f12b9fff700 1 -- 172.21.15.27:0/3746311924 wait complete. 2023-07-06T20:14:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.921+0000 7f12b9fff700 1 Processor -- start 2023-07-06T20:14:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.922+0000 7f12b9fff700 1 -- start start 2023-07-06T20:14:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b9fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12b40b5c90 0x7f12b40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b9fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12b40b6110 0x7f12b40d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b9fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 0x7f12b40d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b9fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40c7730 con 0x7f12b415fca0 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b9fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40c6ab0 con 0x7f12b40b5c90 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b9fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40c6d90 con 0x7f12b40b6110 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 0x7f12b40d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 0x7f12b40d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45094/0 (socket says 172.21.15.27:45094) 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 -- 172.21.15.27:0/2239900615 learned_addr learned my addr 172.21.15.27:0/2239900615 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b37fe700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12b40b5c90 0x7f12b40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b2ffd700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12b40b6110 0x7f12b40d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 -- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12b40b6110 msgr2=0x7f12b40d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12b40b6110 0x7f12b40d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 -- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12b40b5c90 msgr2=0x7f12b40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12b40b5c90 0x7f12b40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b3fff700 1 -- 172.21.15.27:0/2239900615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80094a0 con 0x7f12b415fca0 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.923+0000 7f12b37fe700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12b40b5c90 0x7f12b40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.924+0000 7f12b3fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 0x7f12b40d2f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00efc0 tx=0x7f12ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.924+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00cd40 con 0x7f12b415fca0 2023-07-06T20:14:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.924+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac00f040 con 0x7f12b415fca0 2023-07-06T20:14:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.924+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0146c0 con 0x7f12b415fca0 2023-07-06T20:14:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.924+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b40c7070 con 0x7f12b415fca0 2023-07-06T20:14:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.924+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b41f2020 con 0x7f12b415fca0 2023-07-06T20:14:37.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.925+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b40c7870 con 0x7f12b415fca0 2023-07-06T20:14:37.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.926+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12ac014820 con 0x7f12b415fca0 2023-07-06T20:14:37.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.928+0000 7f129bfff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12a006ace0 0x7f12a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:37.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.928+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12ac08a190 con 0x7f12b415fca0 2023-07-06T20:14:37.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.928+0000 7f12b37fe700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12a006ace0 0x7f12a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:37.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.929+0000 7f12b37fe700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12a006ace0 0x7f12a006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000f60 tx=0x7f12a4007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:37.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:37.932+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f12ac058c30 con 0x7f12b415fca0 2023-07-06T20:14:38.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.143+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12b40008d0 con 0x7f12b415fca0 2023-07-06T20:14:38.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.143+0000 7f129bfff700 1 -- 172.21.15.27:0/2239900615 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12ac00cea0 con 0x7f12b415fca0 2023-07-06T20:14:38.145 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12a006ace0 msgr2=0x7f12a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12a006ace0 0x7f12a006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000f60 tx=0x7f12a4007d50 comp rx=0 tx=0).stop 2023-07-06T20:14:38.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 msgr2=0x7f12b40d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 0x7f12b40d2f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00efc0 tx=0x7f12ac00c3a0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 shutdown_connections 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12a006ace0 0x7f12a006d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b415fca0 0x7f12b40d2f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12b40b5c90 0x7f12b40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 --2- 172.21.15.27:0/2239900615 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f12b40b6110 0x7f12b40d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.145+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 >> 172.21.15.27:0/2239900615 conn(0x7f12b406c150 msgr2=0x7f12b40d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:38.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.146+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 shutdown_connections 2023-07-06T20:14:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.146+0000 7f12b9fff700 1 -- 172.21.15.27:0/2239900615 wait complete. 2023-07-06T20:14:38.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:38.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.369+0000 7fbf139ff700 1 Processor -- start 2023-07-06T20:14:38.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.370+0000 7fbf139ff700 1 -- start start 2023-07-06T20:14:38.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf139ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbf0c15fca0 0x7fbf0c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf139ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b6110 0x7fbf0c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf139ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c0cf780 con 0x7fbf0c0b5c90 2023-07-06T20:14:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf139ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0c0cd490 con 0x7fbf0c15fca0 2023-07-06T20:14:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf139ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c0cd770 con 0x7fbf0c0b6110 2023-07-06T20:14:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf10f9a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbf0c0b5c90 0x7fbf0c0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:32792/0 (socket says 172.21.15.27:32792) 2023-07-06T20:14:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf10f9a700 1 -- 172.21.15.27:0/3228960763 learned_addr learned my addr 172.21.15.27:0/3228960763 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:38.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf1179b700 1 --2- 172.21.15.27:0/3228960763 >> v2:172.21.15.153:3300/0 conn(0x7fbf0c15fca0 0x7fbf0c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf11f9c700 1 --2- 172.21.15.27:0/3228960763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b6110 0x7fbf0c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.371+0000 7fbf03fff700 1 -- 172.21.15.27:0/3228960763 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926819874 0 0) 0x7fbf0c0cf780 con 0x7fbf0c0b5c90 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf03fff700 1 -- 172.21.15.27:0/3228960763 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef80035b0 con 0x7fbf0c0b5c90 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf11f9c700 1 -- 172.21.15.27:0/3228960763 >> v2:172.21.15.153:3300/0 conn(0x7fbf0c15fca0 msgr2=0x7fbf0c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf11f9c700 1 --2- 172.21.15.27:0/3228960763 >> v2:172.21.15.153:3300/0 conn(0x7fbf0c15fca0 0x7fbf0c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf11f9c700 1 -- 172.21.15.27:0/3228960763 >> v1:172.21.15.27:6789/0 conn(0x7fbf0c0b5c90 legacy=0x7fbf0c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf11f9c700 1 -- 172.21.15.27:0/3228960763 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0cda50 con 0x7fbf0c0b6110 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf1179b700 1 --2- 172.21.15.27:0/3228960763 >> v2:172.21.15.153:3300/0 conn(0x7fbf0c15fca0 0x7fbf0c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf11f9c700 1 --2- 172.21.15.27:0/3228960763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b6110 0x7fbf0c0cfc10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009d90 tx=0x7fbf0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99b82dbbaed332c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf03fff700 1 -- 172.21.15.27:0/3228960763 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400e070 con 0x7fbf0c0b6110 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf03fff700 1 -- 172.21.15.27:0/3228960763 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0400bb20 con 0x7fbf0c0b6110 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.372+0000 7fbf03fff700 1 -- 172.21.15.27:0/3228960763 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04012430 con 0x7fbf0c0b6110 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 -- 172.21.15.27:0/3228960763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b6110 msgr2=0x7fbf0c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 --2- 172.21.15.27:0/3228960763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b6110 0x7fbf0c0cfc10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009d90 tx=0x7fbf0400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 -- 172.21.15.27:0/3228960763 shutdown_connections 2023-07-06T20:14:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 --2- 172.21.15.27:0/3228960763 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b6110 0x7fbf0c0cfc10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 --2- 172.21.15.27:0/3228960763 >> v2:172.21.15.153:3300/0 conn(0x7fbf0c15fca0 0x7fbf0c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 -- 172.21.15.27:0/3228960763 >> 172.21.15.27:0/3228960763 conn(0x7fbf0c06c150 msgr2=0x7fbf0c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 -- 172.21.15.27:0/3228960763 shutdown_connections 2023-07-06T20:14:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.373+0000 7fbf139ff700 1 -- 172.21.15.27:0/3228960763 wait complete. 2023-07-06T20:14:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.375+0000 7fbf139ff700 1 Processor -- start 2023-07-06T20:14:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.375+0000 7fbf139ff700 1 -- start start 2023-07-06T20:14:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf139ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b5c90 0x7fbf0c1f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf139ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf0c0b6110 0x7fbf0c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf139ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 0x7fbf0c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf139ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c14c470 con 0x7fbf0c15fca0 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf139ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c14c750 con 0x7fbf0c0b6110 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf139ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c14ca30 con 0x7fbf0c0b5c90 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf11f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 0x7fbf0c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.376+0000 7fbf11f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 0x7fbf0c159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45110/0 (socket says 172.21.15.27:45110) 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 -- 172.21.15.27:0/318049611 learned_addr learned my addr 172.21.15.27:0/318049611 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf1179b700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b5c90 0x7fbf0c1f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf10f9a700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf0c0b6110 0x7fbf0c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 -- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b5c90 msgr2=0x7fbf0c1f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b5c90 0x7fbf0c1f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 -- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf0c0b6110 msgr2=0x7fbf0c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf0c0b6110 0x7fbf0c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 -- 172.21.15.27:0/318049611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080094a0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf11f9c700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 0x7fbf0c159dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf04014040 tx=0x7fbf04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.377+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040129f0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.378+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04020070 con 0x7fbf0c15fca0 2023-07-06T20:14:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.378+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04012db0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.378+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf04009a70 con 0x7fbf0c15fca0 2023-07-06T20:14:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.378+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c14cff0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.379+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c0c7bc0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.380+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf04004040 con 0x7fbf0c15fca0 2023-07-06T20:14:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.381+0000 7fbf01ffb700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbef006ace0 0x7fbef006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.381+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf04022070 con 0x7fbf0c15fca0 2023-07-06T20:14:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.381+0000 7fbf1179b700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbef006ace0 0x7fbef006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.382+0000 7fbf1179b700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbef006ace0 0x7fbef006d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009c60 tx=0x7fbf08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.385+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf0c0c7bc0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.590+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf0c0c7bc0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.590+0000 7fbf01ffb700 1 -- 172.21.15.27:0/318049611 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf0c0c7bc0 con 0x7fbf0c15fca0 2023-07-06T20:14:38.592 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:38.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbef006ace0 msgr2=0x7fbef006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbef006ace0 0x7fbef006d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009c60 tx=0x7fbf08008040 comp rx=0 tx=0).stop 2023-07-06T20:14:38.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 msgr2=0x7fbf0c159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 0x7fbf0c159dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf04014040 tx=0x7fbf04004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 shutdown_connections 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbef006ace0 0x7fbef006d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf0c15fca0 0x7fbf0c159dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf0c0b5c90 0x7fbf0c1f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 --2- 172.21.15.27:0/318049611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf0c0b6110 0x7fbf0c0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 >> 172.21.15.27:0/318049611 conn(0x7fbf0c06c150 msgr2=0x7fbf0c15ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:38.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 shutdown_connections 2023-07-06T20:14:38.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.592+0000 7fbf139ff700 1 -- 172.21.15.27:0/318049611 wait complete. 2023-07-06T20:14:38.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:38.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.814+0000 7f690bfff700 1 Processor -- start 2023-07-06T20:14:38.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.814+0000 7f690bfff700 1 -- start start 2023-07-06T20:14:38.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.815+0000 7f690bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f690415fca0 0x7f69040d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f690bfff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040cf780 con 0x7f69040b6110 2023-07-06T20:14:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f690bfff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040cd490 con 0x7f69040b5c90 2023-07-06T20:14:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f690bfff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69040cd770 con 0x7f690415fca0 2023-07-06T20:14:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f690a59c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f69040b6110 0x7f69040cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:32794/0 (socket says 172.21.15.27:32794) 2023-07-06T20:14:38.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f690a59c700 1 -- 172.21.15.27:0/1676758067 learned_addr learned my addr 172.21.15.27:0/1676758067 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f6909d9b700 1 --2- 172.21.15.27:0/1676758067 >> v2:172.21.15.153:3300/0 conn(0x7f690415fca0 0x7f69040d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f6908d99700 1 -- 172.21.15.27:0/1676758067 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939751455 0 0) 0x7f69040cf780 con 0x7f69040b6110 2023-07-06T20:14:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.816+0000 7f6908d99700 1 -- 172.21.15.27:0/1676758067 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f00035b0 con 0x7f69040b6110 2023-07-06T20:14:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6909d9b700 1 -- 172.21.15.27:0/1676758067 >> v1:172.21.15.153:6789/0 conn(0x7f69040b5c90 legacy=0x7f69040d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6909d9b700 1 -- 172.21.15.27:0/1676758067 >> v1:172.21.15.27:6789/0 conn(0x7f69040b6110 legacy=0x7f69040cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6909d9b700 1 -- 172.21.15.27:0/1676758067 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040cda50 con 0x7f690415fca0 2023-07-06T20:14:38.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6909d9b700 1 --2- 172.21.15.27:0/1676758067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f690415fca0 0x7f69040d1420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6900009840 tx=0x7f6900009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78170f5203832721 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:38.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6908d99700 1 -- 172.21.15.27:0/1676758067 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011830 con 0x7f690415fca0 2023-07-06T20:14:38.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6908d99700 1 -- 172.21.15.27:0/1676758067 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900011990 con 0x7f690415fca0 2023-07-06T20:14:38.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.817+0000 7f6908d99700 1 -- 172.21.15.27:0/1676758067 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011d20 con 0x7f690415fca0 2023-07-06T20:14:38.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 -- 172.21.15.27:0/1676758067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f690415fca0 msgr2=0x7f69040d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 --2- 172.21.15.27:0/1676758067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f690415fca0 0x7f69040d1420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6900009840 tx=0x7f6900009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:38.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 -- 172.21.15.27:0/1676758067 shutdown_connections 2023-07-06T20:14:38.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 --2- 172.21.15.27:0/1676758067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f690415fca0 0x7f69040d1420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 -- 172.21.15.27:0/1676758067 >> 172.21.15.27:0/1676758067 conn(0x7f690406c150 msgr2=0x7f690406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:38.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 -- 172.21.15.27:0/1676758067 shutdown_connections 2023-07-06T20:14:38.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.818+0000 7f690bfff700 1 -- 172.21.15.27:0/1676758067 wait complete. 2023-07-06T20:14:38.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.820+0000 7f690bfff700 1 Processor -- start 2023-07-06T20:14:38.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.820+0000 7f690bfff700 1 -- start start 2023-07-06T20:14:38.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69040b5c90 0x7f69040ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 0x7f69040cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f690415fca0 0x7f690414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690bfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690414db50 con 0x7f69040b6110 2023-07-06T20:14:38.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690bfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040c69b0 con 0x7f69040b5c90 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690bfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040caf00 con 0x7f690415fca0 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690959a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 0x7f69040cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690959a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 0x7f69040cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45122/0 (socket says 172.21.15.27:45122) 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690959a700 1 -- 172.21.15.27:0/2415009170 learned_addr learned my addr 172.21.15.27:0/2415009170 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f6909d9b700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69040b5c90 0x7f69040ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690a59c700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f690415fca0 0x7f690414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690959a700 1 -- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f690415fca0 msgr2=0x7f690414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.821+0000 7f690959a700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f690415fca0 0x7f690414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f690959a700 1 -- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69040b5c90 msgr2=0x7f69040ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f690959a700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69040b5c90 0x7f69040ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f690959a700 1 -- 172.21.15.27:0/2415009170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000094a0 con 0x7f69040b6110 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f690959a700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 0x7f69040cc950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68f400c4e0 tx=0x7f68f400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40052d0 con 0x7f69040b6110 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4005430 con 0x7f69040b6110 2023-07-06T20:14:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40038c0 con 0x7f69040b6110 2023-07-06T20:14:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69040cb1e0 con 0x7f69040b6110 2023-07-06T20:14:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.822+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69040cb7a0 con 0x7f69040b6110 2023-07-06T20:14:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.823+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69040008d0 con 0x7f69040b6110 2023-07-06T20:14:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.824+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68f4003a20 con 0x7f69040b6110 2023-07-06T20:14:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.825+0000 7f68fa7fc700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68e806ace0 0x7f68e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.825+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68f4088880 con 0x7f69040b6110 2023-07-06T20:14:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.826+0000 7f6909d9b700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68e806ace0 0x7f68e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.826+0000 7f6909d9b700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68e806ace0 0x7f68e806d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69000090b0 tx=0x7f690001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:38.829+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68f4053890 con 0x7f69040b6110 2023-07-06T20:14:39.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.042+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69040d09c0 con 0x7f69040b6110 2023-07-06T20:14:39.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.043+0000 7f68fa7fc700 1 -- 172.21.15.27:0/2415009170 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f68f4056eb0 con 0x7f69040b6110 2023-07-06T20:14:39.046 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:39.046 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:38.958863+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68e806ace0 msgr2=0x7f68e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68e806ace0 0x7f68e806d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69000090b0 tx=0x7f690001c040 comp rx=0 tx=0).stop 2023-07-06T20:14:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 msgr2=0x7f69040cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 0x7f69040cc950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68f400c4e0 tx=0x7f68f400c7f0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 shutdown_connections 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68e806ace0 0x7f68e806d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f690415fca0 0x7f690414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69040b5c90 0x7f69040ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 --2- 172.21.15.27:0/2415009170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69040b6110 0x7f69040cc950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 >> 172.21.15.27:0/2415009170 conn(0x7f690406c150 msgr2=0x7f69040d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 shutdown_connections 2023-07-06T20:14:39.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.046+0000 7f690bfff700 1 -- 172.21.15.27:0/2415009170 wait complete. 2023-07-06T20:14:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:39.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:39.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.272+0000 7f72567ff700 1 Processor -- start 2023-07-06T20:14:39.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.272+0000 7f72567ff700 1 -- start start 2023-07-06T20:14:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.273+0000 7f72567ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f72500d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.273+0000 7f72567ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f72500b5c90 0x7f72500d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.273+0000 7f72567ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f72500b6100 0x7f72500d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.273+0000 7f72567ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72500cf780 con 0x7f72500b5c90 2023-07-06T20:14:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.273+0000 7f72567ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72500cd490 con 0x7f72500b6100 2023-07-06T20:14:39.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.273+0000 7f72567ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72500cd770 con 0x7f725015fca0 2023-07-06T20:14:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f72500b5c90 0x7f72500d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f72500b5c90 0x7f72500d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45134/0 (socket says 172.21.15.27:45134) 2023-07-06T20:14:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 -- 172.21.15.27:0/1876301620 learned_addr learned my addr 172.21.15.27:0/1876301620 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f7254d9c700 1 --2- 172.21.15.27:0/1876301620 >> v2:172.21.15.153:3300/0 conn(0x7f72500b6100 0x7f72500d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724ffff700 1 --2- 172.21.15.27:0/1876301620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f72500d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 -- 172.21.15.27:0/1876301620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 msgr2=0x7f72500d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 --2- 172.21.15.27:0/1876301620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f72500d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 -- 172.21.15.27:0/1876301620 >> v2:172.21.15.153:3300/0 conn(0x7f72500b6100 msgr2=0x7f72500d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 --2- 172.21.15.27:0/1876301620 >> v2:172.21.15.153:3300/0 conn(0x7f72500b6100 0x7f72500d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 -- 172.21.15.27:0/1876301620 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500cda50 con 0x7f72500b5c90 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f7254d9c700 1 --2- 172.21.15.27:0/1876301620 >> v2:172.21.15.153:3300/0 conn(0x7f72500b6100 0x7f72500d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.274+0000 7f724f7fe700 1 --2- 172.21.15.27:0/1876301620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b5c90 0x7f72500d2d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72500b8f80 tx=0x7f7244009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3501e950ccf29e16 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f724effd700 1 -- 172.21.15.27:0/1876301620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400b870 con 0x7f72500b5c90 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f724effd700 1 -- 172.21.15.27:0/1876301620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724400b9d0 con 0x7f72500b5c90 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f724effd700 1 -- 172.21.15.27:0/1876301620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400bd60 con 0x7f72500b5c90 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 -- 172.21.15.27:0/1876301620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b5c90 msgr2=0x7f72500d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 --2- 172.21.15.27:0/1876301620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b5c90 0x7f72500d2d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72500b8f80 tx=0x7f7244009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 -- 172.21.15.27:0/1876301620 shutdown_connections 2023-07-06T20:14:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 --2- 172.21.15.27:0/1876301620 >> v2:172.21.15.153:3300/0 conn(0x7f72500b6100 0x7f72500d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 --2- 172.21.15.27:0/1876301620 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f72500d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 --2- 172.21.15.27:0/1876301620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b5c90 0x7f72500d2d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 -- 172.21.15.27:0/1876301620 >> 172.21.15.27:0/1876301620 conn(0x7f725006c150 msgr2=0x7f725006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.275+0000 7f72567ff700 1 -- 172.21.15.27:0/1876301620 shutdown_connections 2023-07-06T20:14:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.276+0000 7f72567ff700 1 -- 172.21.15.27:0/1876301620 wait complete. 2023-07-06T20:14:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.277+0000 7f72567ff700 1 Processor -- start 2023-07-06T20:14:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.277+0000 7f72567ff700 1 -- start start 2023-07-06T20:14:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.278+0000 7f72567ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72500b5c90 0x7f72500c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.278+0000 7f72567ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 0x7f725014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f72567ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f725014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f72567ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500c9230 con 0x7f72500b6100 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f72567ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b48b0 con 0x7f72500b5c90 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f72567ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b4b90 con 0x7f725015fca0 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 0x7f725014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 0x7f725014af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45136/0 (socket says 172.21.15.27:45136) 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 -- 172.21.15.27:0/1773661611 learned_addr learned my addr 172.21.15.27:0/1773661611 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724ffff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72500b5c90 0x7f72500c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f7254d9c700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f725014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 -- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 msgr2=0x7f725014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f725014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 -- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72500b5c90 msgr2=0x7f72500c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72500b5c90 0x7f72500c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.279+0000 7f724f7fe700 1 -- 172.21.15.27:0/1773661611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244009550 con 0x7f72500b6100 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.280+0000 7f724f7fe700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 0x7f725014af80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.280+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244004310 con 0x7f72500b6100 2023-07-06T20:14:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.280+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724401e070 con 0x7f72500b6100 2023-07-06T20:14:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.280+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440036a0 con 0x7f72500b6100 2023-07-06T20:14:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.280+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72500b4e70 con 0x7f72500b6100 2023-07-06T20:14:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.280+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72500b3b00 con 0x7f72500b6100 2023-07-06T20:14:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.281+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72500008d0 con 0x7f72500b6100 2023-07-06T20:14:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.282+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7244020030 con 0x7f72500b6100 2023-07-06T20:14:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.283+0000 7f724cff9700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723c06ace0 0x7f723c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.284+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f724408a310 con 0x7f72500b6100 2023-07-06T20:14:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.284+0000 7f724ffff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723c06ace0 0x7f723c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.285+0000 7f724ffff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723c06ace0 0x7f723c06d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7240006b20 tx=0x7f7240008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.288+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7244055410 con 0x7f72500b6100 2023-07-06T20:14:39.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.494+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72500d3370 con 0x7f72500b6100 2023-07-06T20:14:39.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.495+0000 7f724cff9700 1 -- 172.21.15.27:0/1773661611 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7244058a30 con 0x7f72500b6100 2023-07-06T20:14:39.497 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.497+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723c06ace0 msgr2=0x7f723c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.497+0000 7f72567ff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723c06ace0 0x7f723c06d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7240006b20 tx=0x7f7240008040 comp rx=0 tx=0).stop 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.497+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 msgr2=0x7f725014af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.497+0000 7f72567ff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 0x7f725014af80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244004750 comp rx=0 tx=0).stop 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 shutdown_connections 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f723c06ace0 0x7f723c06d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f725015fca0 0x7f725014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72500b5c90 0x7f72500c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 --2- 172.21.15.27:0/1773661611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72500b6100 0x7f725014af80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 >> 172.21.15.27:0/1773661611 conn(0x7f725006c150 msgr2=0x7f72500b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 shutdown_connections 2023-07-06T20:14:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.498+0000 7f72567ff700 1 -- 172.21.15.27:0/1773661611 wait complete. 2023-07-06T20:14:39.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.722+0000 7f43ae9ff700 1 Processor -- start 2023-07-06T20:14:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.722+0000 7f43ae9ff700 1 -- start start 2023-07-06T20:14:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.723+0000 7f43ae9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43ae9ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a80cf780 con 0x7f43a80b6100 2023-07-06T20:14:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43ae9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a80cd490 con 0x7f43a80b5c90 2023-07-06T20:14:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43ae9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cd770 con 0x7f43a815fca0 2023-07-06T20:14:39.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43acf9c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f43a80b6100 0x7f43a80cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:32806/0 (socket says 172.21.15.27:32806) 2023-07-06T20:14:39.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43acf9c700 1 -- 172.21.15.27:0/1990821753 learned_addr learned my addr 172.21.15.27:0/1990821753 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:39.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43a7fff700 1 --2- 172.21.15.27:0/1990821753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169535096 0 0) 0x7f43a80cf780 con 0x7f43a80b6100 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43900035b0 con 0x7f43a80b6100 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867987002 0 0) 0x7f43a80cd490 con 0x7f43a80b5c90 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.724+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a80cf780 con 0x7f43a80b5c90 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a7fff700 1 -- 172.21.15.27:0/1990821753 >> v1:172.21.15.153:6789/0 conn(0x7f43a80b5c90 legacy=0x7f43a80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3326323488 0 0) 0x7f43900035b0 con 0x7f43a80b6100 2023-07-06T20:14:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a7fff700 1 -- 172.21.15.27:0/1990821753 >> v1:172.21.15.27:6789/0 conn(0x7f43a80b6100 legacy=0x7f43a80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a7fff700 1 -- 172.21.15.27:0/1990821753 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80cda50 con 0x7f43a815fca0 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a7fff700 1 --2- 172.21.15.27:0/1990821753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80d1420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4398009870 tx=0x7f4398009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16cdd043e91158b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00019f0 con 0x7f43a815fca0 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4398004030 con 0x7f43a815fca0 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.725+0000 7f43a6ffd700 1 -- 172.21.15.27:0/1990821753 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980043a0 con 0x7f43a815fca0 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1990821753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 msgr2=0x7f43a80d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1990821753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80d1420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4398009870 tx=0x7f4398009b80 comp rx=0 tx=0).stop 2023-07-06T20:14:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1990821753 shutdown_connections 2023-07-06T20:14:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1990821753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80d1420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1990821753 >> 172.21.15.27:0/1990821753 conn(0x7f43a806c150 msgr2=0x7f43a806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:39.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1990821753 shutdown_connections 2023-07-06T20:14:39.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.726+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1990821753 wait complete. 2023-07-06T20:14:39.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.728+0000 7f43ae9ff700 1 Processor -- start 2023-07-06T20:14:39.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.729+0000 7f43ae9ff700 1 -- start start 2023-07-06T20:14:39.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43ae9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a80b5c90 0x7f43a80cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43ae9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 0x7f43a81590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43ae9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43ae9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cb440 con 0x7f43a80b6100 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43ae9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a815ebe0 con 0x7f43a80b5c90 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43ae9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cb690 con 0x7f43a815fca0 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 0x7f43a81590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 0x7f43a81590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45150/0 (socket says 172.21.15.27:45150) 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43a77fe700 1 -- 172.21.15.27:0/1234403508 learned_addr learned my addr 172.21.15.27:0/1234403508 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43acf9c700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.730+0000 7f43a7fff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a80b5c90 0x7f43a80cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a77fe700 1 -- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 msgr2=0x7f43a80ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a77fe700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a77fe700 1 -- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a80b5c90 msgr2=0x7f43a80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a77fe700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a80b5c90 0x7f43a80cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a77fe700 1 -- 172.21.15.27:0/1234403508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980094a0 con 0x7f43a80b6100 2023-07-06T20:14:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43acf9c700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a77fe700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 0x7f43a81590e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c520 tx=0x7f439c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:39.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0052d0 con 0x7f43a80b6100 2023-07-06T20:14:39.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c005430 con 0x7f43a80b6100 2023-07-06T20:14:39.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0038c0 con 0x7f43a80b6100 2023-07-06T20:14:39.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a80cb970 con 0x7f43a80b6100 2023-07-06T20:14:39.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.731+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a80cbf30 con 0x7f43a80b6100 2023-07-06T20:14:39.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.732+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a80008d0 con 0x7f43a80b6100 2023-07-06T20:14:39.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.733+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f439c003a20 con 0x7f43a80b6100 2023-07-06T20:14:39.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.734+0000 7f43a4ff9700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f439406ace0 0x7f439406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:39.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.734+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f439c088aa0 con 0x7f43a80b6100 2023-07-06T20:14:39.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.735+0000 7f43a7fff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f439406ace0 0x7f439406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:39.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.736+0000 7f43a7fff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f439406ace0 0x7f439406d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43980090b0 tx=0x7f4398013040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.738+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f439c053b00 con 0x7f43a80b6100 2023-07-06T20:14:39.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.944+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43a80cec70 con 0x7f43a80b6100 2023-07-06T20:14:39.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.944+0000 7f43a4ff9700 1 -- 172.21.15.27:0/1234403508 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f439c017020 con 0x7f43a80b6100 2023-07-06T20:14:39.946 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f439406ace0 msgr2=0x7f439406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f439406ace0 0x7f439406d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43980090b0 tx=0x7f4398013040 comp rx=0 tx=0).stop 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 msgr2=0x7f43a81590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 0x7f43a81590e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c520 tx=0x7f439c00c8e0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 shutdown_connections 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f439406ace0 0x7f439406d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43a815fca0 0x7f43a80ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43a80b5c90 0x7f43a80cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 --2- 172.21.15.27:0/1234403508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a80b6100 0x7f43a81590e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:39.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 >> 172.21.15.27:0/1234403508 conn(0x7f43a806c150 msgr2=0x7f43a806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:39.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 shutdown_connections 2023-07-06T20:14:39.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:39.947+0000 7f43ae9ff700 1 -- 172.21.15.27:0/1234403508 wait complete. 2023-07-06T20:14:39.968 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.180+0000 7fc799bff700 1 Processor -- start 2023-07-06T20:14:40.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.180+0000 7fc799bff700 1 -- start start 2023-07-06T20:14:40.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc799bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc79415fca0 0x7fc7940d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc799bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc7940b5c90 0x7fc7940d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc799bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940cf780 con 0x7fc7940b6110 2023-07-06T20:14:40.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc799bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7940cd490 con 0x7fc79415fca0 2023-07-06T20:14:40.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc799bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7940cd770 con 0x7fc7940b5c90 2023-07-06T20:14:40.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc7937fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc79415fca0 0x7fc7940d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc793fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc7940b6110 0x7fc7940cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:32814/0 (socket says 172.21.15.27:32814) 2023-07-06T20:14:40.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.181+0000 7fc793fff700 1 -- 172.21.15.27:0/475861362 learned_addr learned my addr 172.21.15.27:0/475861362 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc792ffd700 1 --2- 172.21.15.27:0/475861362 >> v2:172.21.15.153:3300/0 conn(0x7fc7940b5c90 0x7fc7940d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7937fe700 1 -- 172.21.15.27:0/475861362 >> v2:172.21.15.153:3300/0 conn(0x7fc7940b5c90 msgr2=0x7fc7940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7937fe700 1 --2- 172.21.15.27:0/475861362 >> v2:172.21.15.153:3300/0 conn(0x7fc7940b5c90 0x7fc7940d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7937fe700 1 -- 172.21.15.27:0/475861362 >> v1:172.21.15.27:6789/0 conn(0x7fc7940b6110 legacy=0x7fc7940cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7937fe700 1 -- 172.21.15.27:0/475861362 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940cda50 con 0x7fc79415fca0 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7927fc700 1 -- 172.21.15.27:0/475861362 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583344920 0 0) 0x7fc7940cf780 con 0x7fc7940b6110 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7937fe700 1 --2- 172.21.15.27:0/475861362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc79415fca0 0x7fc7940d1420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc79406b6f0 tx=0x7fc784009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb62c0bf133137cb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.182+0000 7fc7927fc700 1 -- 172.21.15.27:0/475861362 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784011830 con 0x7fc79415fca0 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc7927fc700 1 -- 172.21.15.27:0/475861362 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784011990 con 0x7fc79415fca0 2023-07-06T20:14:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc7927fc700 1 -- 172.21.15.27:0/475861362 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784011d20 con 0x7fc79415fca0 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 -- 172.21.15.27:0/475861362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc79415fca0 msgr2=0x7fc7940d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 --2- 172.21.15.27:0/475861362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc79415fca0 0x7fc7940d1420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc79406b6f0 tx=0x7fc784009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 -- 172.21.15.27:0/475861362 shutdown_connections 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 --2- 172.21.15.27:0/475861362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc79415fca0 0x7fc7940d1420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 --2- 172.21.15.27:0/475861362 >> v2:172.21.15.153:3300/0 conn(0x7fc7940b5c90 0x7fc7940d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 -- 172.21.15.27:0/475861362 >> 172.21.15.27:0/475861362 conn(0x7fc79406c150 msgr2=0x7fc79406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 -- 172.21.15.27:0/475861362 shutdown_connections 2023-07-06T20:14:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.183+0000 7fc799bff700 1 -- 172.21.15.27:0/475861362 wait complete. 2023-07-06T20:14:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.185+0000 7fc799bff700 1 Processor -- start 2023-07-06T20:14:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.185+0000 7fc799bff700 1 -- start start 2023-07-06T20:14:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc799bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7940b5c90 0x7fc794160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc799bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 0x7fc7941568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc799bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc79415fca0 0x7fc7940cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc799bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794159ac0 con 0x7fc7940b6110 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc799bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940cc760 con 0x7fc79415fca0 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc799bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc794159520 con 0x7fc7940b5c90 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc792ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 0x7fc7941568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc792ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 0x7fc7941568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45176/0 (socket says 172.21.15.27:45176) 2023-07-06T20:14:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc792ffd700 1 -- 172.21.15.27:0/4147452686 learned_addr learned my addr 172.21.15.27:0/4147452686 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.186+0000 7fc793fff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc79415fca0 0x7fc7940cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc7937fe700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7940b5c90 0x7fc794160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc792ffd700 1 -- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7940b5c90 msgr2=0x7fc794160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc792ffd700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7940b5c90 0x7fc794160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc792ffd700 1 -- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc79415fca0 msgr2=0x7fc7940cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc792ffd700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc79415fca0 0x7fc7940cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc792ffd700 1 -- 172.21.15.27:0/4147452686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7840094a0 con 0x7fc7940b6110 2023-07-06T20:14:40.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc793fff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc79415fca0 0x7fc7940cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc7937fe700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7940b5c90 0x7fc794160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc792ffd700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 0x7fc7941568d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e810 tx=0x7fc78800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788013610 con 0x7fc7940b6110 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788013770 con 0x7fc7940b6110 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941597e0 con 0x7fc7940b6110 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.187+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788013b20 con 0x7fc7940b6110 2023-07-06T20:14:40.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.188+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7940cbd00 con 0x7fc7940b6110 2023-07-06T20:14:40.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.189+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7940008d0 con 0x7fc7940b6110 2023-07-06T20:14:40.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.190+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc788013c80 con 0x7fc7940b6110 2023-07-06T20:14:40.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.191+0000 7fc77bfff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc78006ace0 0x7fc78006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.191+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc788089230 con 0x7fc7940b6110 2023-07-06T20:14:40.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.191+0000 7fc7937fe700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc78006ace0 0x7fc78006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.192+0000 7fc7937fe700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc78006ace0 0x7fc78006d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7840090b0 tx=0x7fc78401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:40.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.195+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc788057ca0 con 0x7fc7940b6110 2023-07-06T20:14:40.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.400+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7940d06b0 con 0x7fc7940b6110 2023-07-06T20:14:40.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.401+0000 7fc77bfff700 1 -- 172.21.15.27:0/4147452686 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc788057830 con 0x7fc7940b6110 2023-07-06T20:14:40.403 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:40.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc78006ace0 msgr2=0x7fc78006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc78006ace0 0x7fc78006d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7840090b0 tx=0x7fc78401c040 comp rx=0 tx=0).stop 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 msgr2=0x7fc7941568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 0x7fc7941568d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e810 tx=0x7fc78800ebd0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 shutdown_connections 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc78006ace0 0x7fc78006d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc79415fca0 0x7fc7940cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7940b5c90 0x7fc794160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 --2- 172.21.15.27:0/4147452686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7940b6110 0x7fc7941568d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.403+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 >> 172.21.15.27:0/4147452686 conn(0x7fc79406c150 msgr2=0x7fc7940d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:40.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.404+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 shutdown_connections 2023-07-06T20:14:40.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.404+0000 7fc799bff700 1 -- 172.21.15.27:0/4147452686 wait complete. 2023-07-06T20:14:40.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.629+0000 7fea007ff700 1 Processor -- start 2023-07-06T20:14:40.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.629+0000 7fea007ff700 1 -- start start 2023-07-06T20:14:40.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fea007ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b6110 0x7fe9f80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fea007ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80cf780 con 0x7fe9f80b5c90 2023-07-06T20:14:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fea007ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80cd490 con 0x7fe9f815fca0 2023-07-06T20:14:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fea007ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80cd770 con 0x7fe9f80b6110 2023-07-06T20:14:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fe9fdd9a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9f80b5c90 0x7fe9f80d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:32828/0 (socket says 172.21.15.27:32828) 2023-07-06T20:14:40.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fe9fdd9a700 1 -- 172.21.15.27:0/362435786 learned_addr learned my addr 172.21.15.27:0/362435786 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:40.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fe9fed9c700 1 --2- 172.21.15.27:0/362435786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b6110 0x7fe9f80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575968275 0 0) 0x7fe9f80cf780 con 0x7fe9f80b5c90 2023-07-06T20:14:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.631+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e40035b0 con 0x7fe9f80b5c90 2023-07-06T20:14:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489444403 0 0) 0x7fe9f80cd490 con 0x7fe9f815fca0 2023-07-06T20:14:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f80cf780 con 0x7fe9f815fca0 2023-07-06T20:14:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fed9c700 1 -- 172.21.15.27:0/362435786 >> v1:172.21.15.153:6789/0 conn(0x7fe9f815fca0 legacy=0x7fe9f80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fed9c700 1 -- 172.21.15.27:0/362435786 >> v1:172.21.15.27:6789/0 conn(0x7fe9f80b5c90 legacy=0x7fe9f80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fed9c700 1 -- 172.21.15.27:0/362435786 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80cda50 con 0x7fe9f80b6110 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fed9c700 1 --2- 172.21.15.27:0/362435786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b6110 0x7fe9f80cfc10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009dc0 tx=0x7fe9f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13140923447823fc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000e070 con 0x7fe9f80b6110 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f000bb20 con 0x7fe9f80b6110 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.632+0000 7fe9fd599700 1 -- 172.21.15.27:0/362435786 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0012430 con 0x7fe9f80b6110 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 -- 172.21.15.27:0/362435786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b6110 msgr2=0x7fe9f80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 --2- 172.21.15.27:0/362435786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b6110 0x7fe9f80cfc10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009dc0 tx=0x7fe9f000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 -- 172.21.15.27:0/362435786 shutdown_connections 2023-07-06T20:14:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 --2- 172.21.15.27:0/362435786 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b6110 0x7fe9f80cfc10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 -- 172.21.15.27:0/362435786 >> 172.21.15.27:0/362435786 conn(0x7fe9f806c150 msgr2=0x7fe9f806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 -- 172.21.15.27:0/362435786 shutdown_connections 2023-07-06T20:14:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.633+0000 7fea007ff700 1 -- 172.21.15.27:0/362435786 wait complete. 2023-07-06T20:14:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.635+0000 7fea007ff700 1 Processor -- start 2023-07-06T20:14:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.635+0000 7fea007ff700 1 -- start start 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fea007ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b5c90 0x7fe9f80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fea007ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f80b6110 0x7fe9f80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fea007ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 0x7fe9f8156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fea007ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80cb4a0 con 0x7fe9f815fca0 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fea007ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80b3490 con 0x7fe9f80b6110 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fea007ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80b3770 con 0x7fe9f80b5c90 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fe9fed9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 0x7fe9f8156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fe9fed9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 0x7fe9f8156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45182/0 (socket says 172.21.15.27:45182) 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.636+0000 7fe9fed9c700 1 -- 172.21.15.27:0/3277128564 learned_addr learned my addr 172.21.15.27:0/3277128564 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fe59b700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b5c90 0x7fe9f80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fdd9a700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f80b6110 0x7fe9f80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fed9c700 1 -- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b5c90 msgr2=0x7fe9f80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fed9c700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b5c90 0x7fe9f80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fed9c700 1 -- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f80b6110 msgr2=0x7fe9f80b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fed9c700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f80b6110 0x7fe9f80b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fed9c700 1 -- 172.21.15.27:0/3277128564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009a70 con 0x7fe9f815fca0 2023-07-06T20:14:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fe59b700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b5c90 0x7fe9f80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9fed9c700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 0x7fe9f8156f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000b370 tx=0x7fe9f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f001b810 con 0x7fe9f815fca0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f0020070 con 0x7fe9f815fca0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f001bbf0 con 0x7fe9f815fca0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.637+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80b3a50 con 0x7fe9f815fca0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.638+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f80b4010 con 0x7fe9f815fca0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.639+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f80008d0 con 0x7fe9f815fca0 2023-07-06T20:14:40.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.640+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9f001bd50 con 0x7fe9f815fca0 2023-07-06T20:14:40.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.641+0000 7fe9eeffd700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe9dc06ace0 0x7fe9dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:40.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.641+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe9f0022070 con 0x7fe9f815fca0 2023-07-06T20:14:40.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.641+0000 7fe9fe59b700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe9dc06ace0 0x7fe9dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.642+0000 7fe9fe59b700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe9dc06ace0 0x7fe9dc06d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007fd0 tx=0x7fe9f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:40.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.645+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9f0056980 con 0x7fe9f815fca0 2023-07-06T20:14:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.852+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9f815ada0 con 0x7fe9f815fca0 2023-07-06T20:14:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.852+0000 7fe9eeffd700 1 -- 172.21.15.27:0/3277128564 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9f002c0a0 con 0x7fe9f815fca0 2023-07-06T20:14:40.854 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe9dc06ace0 msgr2=0x7fe9dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe9dc06ace0 0x7fe9dc06d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007fd0 tx=0x7fe9f4009040 comp rx=0 tx=0).stop 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 msgr2=0x7fe9f8156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 0x7fe9f8156f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000b370 tx=0x7fe9f0004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 shutdown_connections 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe9dc06ace0 0x7fe9dc06d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9f815fca0 0x7fe9f8156f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9f80b5c90 0x7fe9f80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 --2- 172.21.15.27:0/3277128564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f80b6110 0x7fe9f80b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 >> 172.21.15.27:0/3277128564 conn(0x7fe9f806c150 msgr2=0x7fe9f80b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 shutdown_connections 2023-07-06T20:14:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:40.855+0000 7fea007ff700 1 -- 172.21.15.27:0/3277128564 wait complete. 2023-07-06T20:14:40.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.080+0000 7f4ed1dff700 1 Processor -- start 2023-07-06T20:14:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.081+0000 7f4ed1dff700 1 -- start start 2023-07-06T20:14:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ed1dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4ecc15fca0 0x7f4ecc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ed1dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b5c90 0x7f4ecc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ed1dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ecc0b60e0 0x7f4ecc0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ed1dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cf780 con 0x7f4ecc0b60e0 2023-07-06T20:14:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ed1dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cd490 con 0x7f4ecc15fca0 2023-07-06T20:14:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ed1dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cd770 con 0x7f4ecc0b5c90 2023-07-06T20:14:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ecc0b60e0 0x7f4ecc0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ecc0b60e0 0x7f4ecc0d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45194/0 (socket says 172.21.15.27:45194) 2023-07-06T20:14:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 -- 172.21.15.27:0/1797287046 learned_addr learned my addr 172.21.15.27:0/1797287046 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 -- 172.21.15.27:0/1797287046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b5c90 msgr2=0x7f4ecc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 --2- 172.21.15.27:0/1797287046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b5c90 0x7f4ecc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 -- 172.21.15.27:0/1797287046 >> v2:172.21.15.153:3300/0 conn(0x7f4ecc15fca0 msgr2=0x7f4ecc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 --2- 172.21.15.27:0/1797287046 >> v2:172.21.15.153:3300/0 conn(0x7f4ecc15fca0 0x7f4ecc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.082+0000 7f4ecbfff700 1 -- 172.21.15.27:0/1797287046 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0cda50 con 0x7f4ecc0b60e0 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.083+0000 7f4ecbfff700 1 --2- 172.21.15.27:0/1797287046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc0b60e0 0x7f4ecc0d3280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0d0930 tx=0x7f4ec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471306dc4803df18 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.083+0000 7f4eca7fc700 1 -- 172.21.15.27:0/1797287046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400e070 con 0x7f4ecc0b60e0 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.083+0000 7f4eca7fc700 1 -- 172.21.15.27:0/1797287046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec400b9c0 con 0x7f4ecc0b60e0 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.083+0000 7f4eca7fc700 1 -- 172.21.15.27:0/1797287046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400bd70 con 0x7f4ecc0b60e0 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.083+0000 7f4ed1dff700 1 -- 172.21.15.27:0/1797287046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc0b60e0 msgr2=0x7f4ecc0d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.083+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/1797287046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc0b60e0 0x7f4ecc0d3280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0d0930 tx=0x7f4ec400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 -- 172.21.15.27:0/1797287046 shutdown_connections 2023-07-06T20:14:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/1797287046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc0b60e0 0x7f4ecc0d3280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/1797287046 >> v2:172.21.15.153:3300/0 conn(0x7f4ecc15fca0 0x7f4ecc0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/1797287046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b5c90 0x7f4ecc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 -- 172.21.15.27:0/1797287046 >> 172.21.15.27:0/1797287046 conn(0x7f4ecc06c150 msgr2=0x7f4ecc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 -- 172.21.15.27:0/1797287046 shutdown_connections 2023-07-06T20:14:41.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.084+0000 7f4ed1dff700 1 -- 172.21.15.27:0/1797287046 wait complete. 2023-07-06T20:14:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.086+0000 7f4ed1dff700 1 Processor -- start 2023-07-06T20:14:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.086+0000 7f4ed1dff700 1 -- start start 2023-07-06T20:14:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ed1dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ecc0b5c90 0x7f4ecc15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ed1dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b60e0 0x7f4ecc0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ed1dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 0x7f4ecc0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ed1dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1f3070 con 0x7f4ecc15fca0 2023-07-06T20:14:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ed1dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc15c9b0 con 0x7f4ecc0b5c90 2023-07-06T20:14:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ed1dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc15b9b0 con 0x7f4ecc0b60e0 2023-07-06T20:14:41.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 0x7f4ecc0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 0x7f4ecc0cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45200/0 (socket says 172.21.15.27:45200) 2023-07-06T20:14:41.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 -- 172.21.15.27:0/291434311 learned_addr learned my addr 172.21.15.27:0/291434311 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:41.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 -- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b60e0 msgr2=0x7f4ecc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b60e0 0x7f4ecc0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 -- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ecc0b5c90 msgr2=0x7f4ecc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ecc0b5c90 0x7f4ecc15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.088+0000 7f4ecbfff700 1 -- 172.21.15.27:0/291434311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4009b30 con 0x7f4ecc15fca0 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.089+0000 7f4ecbfff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 0x7f4ecc0cbd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b370 tx=0x7f4ec4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.089+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4004310 con 0x7f4ecc15fca0 2023-07-06T20:14:41.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.089+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec4004490 con 0x7f4ecc15fca0 2023-07-06T20:14:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.089+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc1f2d90 con 0x7f4ecc15fca0 2023-07-06T20:14:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.089+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4004840 con 0x7f4ecc15fca0 2023-07-06T20:14:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.089+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc15bf50 con 0x7f4ecc15fca0 2023-07-06T20:14:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.090+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0008d0 con 0x7f4ecc15fca0 2023-07-06T20:14:41.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.091+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ec401c440 con 0x7f4ecc15fca0 2023-07-06T20:14:41.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.093+0000 7f4eb3fff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4eb806ace0 0x7f4eb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.093+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4ec408ab70 con 0x7f4ecc15fca0 2023-07-06T20:14:41.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.093+0000 7f4ecb7fe700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4eb806ace0 0x7f4eb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.094+0000 7f4ecb7fe700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4eb806ace0 0x7f4eb806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009710 tx=0x7f4ebc006c80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.098+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ec4055c70 con 0x7f4ecc15fca0 2023-07-06T20:14:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.303+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ecc0d06b0 con 0x7f4ecc15fca0 2023-07-06T20:14:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.305+0000 7f4eb3fff700 1 -- 172.21.15.27:0/291434311 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f4ec4020090 con 0x7f4ecc15fca0 2023-07-06T20:14:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:41.307 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4eb806ace0 msgr2=0x7f4eb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4eb806ace0 0x7f4eb806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009710 tx=0x7f4ebc006c80 comp rx=0 tx=0).stop 2023-07-06T20:14:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 msgr2=0x7f4ecc0cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 0x7f4ecc0cbd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b370 tx=0x7f4ec4004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 shutdown_connections 2023-07-06T20:14:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4eb806ace0 0x7f4eb806d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ecc15fca0 0x7f4ecc0cbd50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ecc0b5c90 0x7f4ecc15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 --2- 172.21.15.27:0/291434311 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4ecc0b60e0 0x7f4ecc0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 >> 172.21.15.27:0/291434311 conn(0x7f4ecc06c150 msgr2=0x7f4ecc0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 shutdown_connections 2023-07-06T20:14:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.307+0000 7f4ed1dff700 1 -- 172.21.15.27:0/291434311 wait complete. 2023-07-06T20:14:41.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:41.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:41.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.532+0000 7feb491ff700 1 Processor -- start 2023-07-06T20:14:41.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.532+0000 7feb491ff700 1 -- start start 2023-07-06T20:14:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.533+0000 7feb491ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb440d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.533+0000 7feb491ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb440b5c90 0x7feb440d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.533+0000 7feb491ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7feb440b6100 0x7feb440d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.533+0000 7feb491ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb440cf780 con 0x7feb440b5c90 2023-07-06T20:14:41.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.533+0000 7feb491ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb440cd490 con 0x7feb440b6100 2023-07-06T20:14:41.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.533+0000 7feb491ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb440cd770 con 0x7feb4415fca0 2023-07-06T20:14:41.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb440b5c90 0x7feb440d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb440b5c90 0x7feb440d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45202/0 (socket says 172.21.15.27:45202) 2023-07-06T20:14:41.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 -- 172.21.15.27:0/1201219857 learned_addr learned my addr 172.21.15.27:0/1201219857 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb42d9d700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb440d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4359e700 1 --2- 172.21.15.27:0/1201219857 >> v2:172.21.15.153:3300/0 conn(0x7feb440b6100 0x7feb440d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 -- 172.21.15.27:0/1201219857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 msgr2=0x7feb440d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb440d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 -- 172.21.15.27:0/1201219857 >> v2:172.21.15.153:3300/0 conn(0x7feb440b6100 msgr2=0x7feb440d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 --2- 172.21.15.27:0/1201219857 >> v2:172.21.15.153:3300/0 conn(0x7feb440b6100 0x7feb440d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 -- 172.21.15.27:0/1201219857 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440cda50 con 0x7feb440b5c90 2023-07-06T20:14:41.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4359e700 1 --2- 172.21.15.27:0/1201219857 >> v2:172.21.15.153:3300/0 conn(0x7feb440b6100 0x7feb440d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb42d9d700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.534+0000 7feb4259c700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb440d2d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb440b8f80 tx=0x7feb38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1077dd3e1b7c9e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb41d9b700 1 -- 172.21.15.27:0/1201219857 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800b870 con 0x7feb440b5c90 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb41d9b700 1 -- 172.21.15.27:0/1201219857 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3800b9d0 con 0x7feb440b5c90 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb41d9b700 1 -- 172.21.15.27:0/1201219857 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800bd60 con 0x7feb440b5c90 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 -- 172.21.15.27:0/1201219857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 msgr2=0x7feb440d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb440d2d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb440b8f80 tx=0x7feb38009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 -- 172.21.15.27:0/1201219857 shutdown_connections 2023-07-06T20:14:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 --2- 172.21.15.27:0/1201219857 >> v2:172.21.15.153:3300/0 conn(0x7feb440b6100 0x7feb440d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 --2- 172.21.15.27:0/1201219857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb440d2d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 -- 172.21.15.27:0/1201219857 >> 172.21.15.27:0/1201219857 conn(0x7feb4406c150 msgr2=0x7feb4406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 -- 172.21.15.27:0/1201219857 shutdown_connections 2023-07-06T20:14:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.535+0000 7feb491ff700 1 -- 172.21.15.27:0/1201219857 wait complete. 2023-07-06T20:14:41.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.537+0000 7feb491ff700 1 Processor -- start 2023-07-06T20:14:41.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.537+0000 7feb491ff700 1 -- start start 2023-07-06T20:14:41.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.538+0000 7feb491ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb441579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.538+0000 7feb491ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb440b6100 0x7feb4414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.538+0000 7feb491ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb441e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.538+0000 7feb491ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44157780 con 0x7feb440b5c90 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.538+0000 7feb491ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb441e6ff0 con 0x7feb440b6100 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.538+0000 7feb491ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb441e72d0 con 0x7feb4415fca0 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb441579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb441579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45212/0 (socket says 172.21.15.27:45212) 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 -- 172.21.15.27:0/3892436260 learned_addr learned my addr 172.21.15.27:0/3892436260 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb4259c700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb440b6100 0x7feb4414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb4359e700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb441e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 -- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 msgr2=0x7feb441e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb441e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 -- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb440b6100 msgr2=0x7feb4414a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb440b6100 0x7feb4414a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 -- 172.21.15.27:0/3892436260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38009550 con 0x7feb440b5c90 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.539+0000 7feb42d9d700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb441579a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e810 tx=0x7feb3400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.540+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400c070 con 0x7feb440b5c90 2023-07-06T20:14:41.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.540+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34004230 con 0x7feb440b5c90 2023-07-06T20:14:41.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.540+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340036a0 con 0x7feb440b5c90 2023-07-06T20:14:41.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.540+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb441e75b0 con 0x7feb440b5c90 2023-07-06T20:14:41.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.540+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4415c8e0 con 0x7feb440b5c90 2023-07-06T20:14:41.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.541+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb440008d0 con 0x7feb440b5c90 2023-07-06T20:14:41.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.542+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb34003800 con 0x7feb440b5c90 2023-07-06T20:14:41.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.543+0000 7feb2b7fe700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7feb3006ace0 0x7feb3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.543+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7feb340891d0 con 0x7feb440b5c90 2023-07-06T20:14:41.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.543+0000 7feb4259c700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7feb3006ace0 0x7feb3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.544+0000 7feb4259c700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7feb3006ace0 0x7feb3006d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb38014040 tx=0x7feb38008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.547+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb34054230 con 0x7feb440b5c90 2023-07-06T20:14:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.756+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb440b6f60 con 0x7feb440b5c90 2023-07-06T20:14:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.756+0000 7feb2b7fe700 1 -- 172.21.15.27:0/3892436260 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb34057850 con 0x7feb440b5c90 2023-07-06T20:14:41.758 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:41.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7feb3006ace0 msgr2=0x7feb3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7feb3006ace0 0x7feb3006d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb38014040 tx=0x7feb38008030 comp rx=0 tx=0).stop 2023-07-06T20:14:41.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 msgr2=0x7feb441579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb441579a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e810 tx=0x7feb3400eb20 comp rx=0 tx=0).stop 2023-07-06T20:14:41.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 shutdown_connections 2023-07-06T20:14:41.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7feb3006ace0 0x7feb3006d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb4415fca0 0x7feb441e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb440b5c90 0x7feb441579a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 --2- 172.21.15.27:0/3892436260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb440b6100 0x7feb4414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 >> 172.21.15.27:0/3892436260 conn(0x7feb4406c150 msgr2=0x7feb440d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:41.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 shutdown_connections 2023-07-06T20:14:41.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.759+0000 7feb491ff700 1 -- 172.21.15.27:0/3892436260 wait complete. 2023-07-06T20:14:41.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:41.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.983+0000 7f950a1ff700 1 Processor -- start 2023-07-06T20:14:41.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.983+0000 7f950a1ff700 1 -- start start 2023-07-06T20:14:41.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f950a1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95040b5c90 0x7f95040d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b6110 0x7f95040d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f950a1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040cf780 con 0x7f950415fca0 2023-07-06T20:14:41.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f950a1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95040cd490 con 0x7f95040b5c90 2023-07-06T20:14:41.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f950a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040cd770 con 0x7f95040b6110 2023-07-06T20:14:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f9502ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95040b5c90 0x7f95040d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f95037fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f950415fca0 0x7f95040d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:45424/0 (socket says 172.21.15.27:45424) 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f95037fe700 1 -- 172.21.15.27:0/484714073 learned_addr learned my addr 172.21.15.27:0/484714073 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f9503fff700 1 --2- 172.21.15.27:0/484714073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b6110 0x7f95040d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f9502ffd700 1 -- 172.21.15.27:0/484714073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b6110 msgr2=0x7f95040d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f9502ffd700 1 --2- 172.21.15.27:0/484714073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b6110 0x7f95040d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f9502ffd700 1 -- 172.21.15.27:0/484714073 >> v1:172.21.15.153:6789/0 conn(0x7f950415fca0 legacy=0x7f95040d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.985+0000 7f9502ffd700 1 -- 172.21.15.27:0/484714073 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040cda50 con 0x7f95040b5c90 2023-07-06T20:14:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.986+0000 7f9502ffd700 1 --2- 172.21.15.27:0/484714073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b5c90 0x7f95040d2d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009840 tx=0x7f94f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a24f68278d465b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.986+0000 7f95027fc700 1 -- 172.21.15.27:0/484714073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8011830 con 0x7f95040b5c90 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.986+0000 7f95027fc700 1 -- 172.21.15.27:0/484714073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f8011990 con 0x7f95040b5c90 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.986+0000 7f95027fc700 1 -- 172.21.15.27:0/484714073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8011d20 con 0x7f95040b5c90 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.986+0000 7f950a1ff700 1 -- 172.21.15.27:0/484714073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b5c90 msgr2=0x7f95040d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.986+0000 7f950a1ff700 1 --2- 172.21.15.27:0/484714073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b5c90 0x7f95040d2d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009840 tx=0x7f94f8009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.987+0000 7f950a1ff700 1 -- 172.21.15.27:0/484714073 shutdown_connections 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.987+0000 7f950a1ff700 1 --2- 172.21.15.27:0/484714073 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b6110 0x7f95040d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.987+0000 7f950a1ff700 1 --2- 172.21.15.27:0/484714073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b5c90 0x7f95040d2d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.987+0000 7f950a1ff700 1 -- 172.21.15.27:0/484714073 >> 172.21.15.27:0/484714073 conn(0x7f950406c150 msgr2=0x7f950406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.987+0000 7f950a1ff700 1 -- 172.21.15.27:0/484714073 shutdown_connections 2023-07-06T20:14:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.987+0000 7f950a1ff700 1 -- 172.21.15.27:0/484714073 wait complete. 2023-07-06T20:14:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.988+0000 7f950a1ff700 1 Processor -- start 2023-07-06T20:14:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.988+0000 7f950a1ff700 1 -- start start 2023-07-06T20:14:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.989+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b5c90 0x7f9504157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.989+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 0x7f95040c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.989+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f950415fca0 0x7f95041591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.989+0000 7f950a1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95040cbdb0 con 0x7f95040b6110 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.989+0000 7f950a1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9504157a60 con 0x7f950415fca0 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.989+0000 7f950a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040cbfc0 con 0x7f95040b5c90 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 0x7f95040c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 0x7f95040c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46352/0 (socket says 172.21.15.27:46352) 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 -- 172.21.15.27:0/3700452228 learned_addr learned my addr 172.21.15.27:0/3700452228 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f95037fe700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b5c90 0x7f9504157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9503fff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f950415fca0 0x7f95041591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 -- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b5c90 msgr2=0x7f9504157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b5c90 0x7f9504157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 -- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f950415fca0 msgr2=0x7f95041591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f950415fca0 0x7f95041591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 -- 172.21.15.27:0/3700452228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80094a0 con 0x7f95040b6110 2023-07-06T20:14:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9503fff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f950415fca0 0x7f95041591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.990+0000 7f9502ffd700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 0x7f95040c8700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009fd0 tx=0x7f94f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.991+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f80036a0 con 0x7f95040b6110 2023-07-06T20:14:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.991+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f801f070 con 0x7f95040b6110 2023-07-06T20:14:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.991+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8003a60 con 0x7f95040b6110 2023-07-06T20:14:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.991+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95040cc2a0 con 0x7f95040b6110 2023-07-06T20:14:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.991+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95040cc860 con 0x7f95040b6110 2023-07-06T20:14:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.992+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95040008d0 con 0x7f95040b6110 2023-07-06T20:14:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.993+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94f8003bc0 con 0x7f95040b6110 2023-07-06T20:14:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.994+0000 7f94ebfff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94f006ace0 0x7f94f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.994+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f94f8021070 con 0x7f95040b6110 2023-07-06T20:14:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.995+0000 7f95037fe700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94f006ace0 0x7f94f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.995+0000 7f95037fe700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94f006ace0 0x7f94f006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94f4005910 tx=0x7f94f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:41.998+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94f8055ed0 con 0x7f95040b6110 2023-07-06T20:14:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.206+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f950415a9b0 con 0x7f95040b6110 2023-07-06T20:14:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.207+0000 7f94ebfff700 1 -- 172.21.15.27:0/3700452228 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f94f80594f0 con 0x7f95040b6110 2023-07-06T20:14:42.209 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94f006ace0 msgr2=0x7f94f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94f006ace0 0x7f94f006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94f4005910 tx=0x7f94f4009040 comp rx=0 tx=0).stop 2023-07-06T20:14:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 msgr2=0x7f95040c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 0x7f95040c8700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009fd0 tx=0x7f94f8004750 comp rx=0 tx=0).stop 2023-07-06T20:14:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 shutdown_connections 2023-07-06T20:14:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94f006ace0 0x7f94f006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f950415fca0 0x7f95041591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95040b5c90 0x7f9504157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 --2- 172.21.15.27:0/3700452228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95040b6110 0x7f95040c8700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 >> 172.21.15.27:0/3700452228 conn(0x7f950406c150 msgr2=0x7f95040b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 shutdown_connections 2023-07-06T20:14:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.209+0000 7f950a1ff700 1 -- 172.21.15.27:0/3700452228 wait complete. 2023-07-06T20:14:42.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:42.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.433+0000 7fc1adbff700 1 Processor -- start 2023-07-06T20:14:42.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.434+0000 7fc1adbff700 1 -- start start 2023-07-06T20:14:42.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1adbff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1a80b5c90 0x7fc1a80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1adbff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1a80b6110 0x7fc1a80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1adbff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a80cf780 con 0x7fc1a815fca0 2023-07-06T20:14:42.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1adbff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a80cd490 con 0x7fc1a80b6110 2023-07-06T20:14:42.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1adbff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a80cd770 con 0x7fc1a80b5c90 2023-07-06T20:14:42.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1a7fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1a80b6110 0x7fc1a80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1a77fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc1a815fca0 0x7fc1a80d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56694/0 (socket says 172.21.15.27:56694) 2023-07-06T20:14:42.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc1a77fe700 1 -- 172.21.15.27:0/1324652705 learned_addr learned my addr 172.21.15.27:0/1324652705 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:42.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.435+0000 7fc19ffff700 1 --2- 172.21.15.27:0/1324652705 >> v2:172.21.15.153:3300/0 conn(0x7fc1a80b5c90 0x7fc1a80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a6ffd700 1 -- 172.21.15.27:0/1324652705 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566853814 0 0) 0x7fc1a80cf780 con 0x7fc1a815fca0 2023-07-06T20:14:42.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a7fff700 1 -- 172.21.15.27:0/1324652705 >> v2:172.21.15.153:3300/0 conn(0x7fc1a80b5c90 msgr2=0x7fc1a80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:42.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/1324652705 >> v2:172.21.15.153:3300/0 conn(0x7fc1a80b5c90 0x7fc1a80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a7fff700 1 -- 172.21.15.27:0/1324652705 >> v1:172.21.15.27:6789/0 conn(0x7fc1a815fca0 legacy=0x7fc1a80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a7fff700 1 -- 172.21.15.27:0/1324652705 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a80cda50 con 0x7fc1a80b6110 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc19ffff700 1 --2- 172.21.15.27:0/1324652705 >> v2:172.21.15.153:3300/0 conn(0x7fc1a80b5c90 0x7fc1a80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/1324652705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a80b6110 0x7fc1a80d3280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc1a806b6f0 tx=0x7fc1a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a97b2d6505ed907 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a6ffd700 1 -- 172.21.15.27:0/1324652705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000e070 con 0x7fc1a80b6110 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a6ffd700 1 -- 172.21.15.27:0/1324652705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a000bbd0 con 0x7fc1a80b6110 2023-07-06T20:14:42.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.436+0000 7fc1a6ffd700 1 -- 172.21.15.27:0/1324652705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0012430 con 0x7fc1a80b6110 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 -- 172.21.15.27:0/1324652705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a80b6110 msgr2=0x7fc1a80d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 --2- 172.21.15.27:0/1324652705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a80b6110 0x7fc1a80d3280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc1a806b6f0 tx=0x7fc1a000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 -- 172.21.15.27:0/1324652705 shutdown_connections 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 --2- 172.21.15.27:0/1324652705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a80b6110 0x7fc1a80d3280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 --2- 172.21.15.27:0/1324652705 >> v2:172.21.15.153:3300/0 conn(0x7fc1a80b5c90 0x7fc1a80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 -- 172.21.15.27:0/1324652705 >> 172.21.15.27:0/1324652705 conn(0x7fc1a806c150 msgr2=0x7fc1a806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 -- 172.21.15.27:0/1324652705 shutdown_connections 2023-07-06T20:14:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.437+0000 7fc1adbff700 1 -- 172.21.15.27:0/1324652705 wait complete. 2023-07-06T20:14:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.439+0000 7fc1adbff700 1 Processor -- start 2023-07-06T20:14:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.439+0000 7fc1adbff700 1 -- start start 2023-07-06T20:14:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1adbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1a80b5c90 0x7fc1a8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1adbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a80b6110 0x7fc1a8149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1adbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 0x7fc1a80c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1adbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8158ba0 con 0x7fc1a815fca0 2023-07-06T20:14:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1adbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8157cd0 con 0x7fc1a80b6110 2023-07-06T20:14:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1adbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8157f20 con 0x7fc1a80b5c90 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 0x7fc1a80c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 0x7fc1a80c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46380/0 (socket says 172.21.15.27:46380) 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 -- 172.21.15.27:0/2962610024 learned_addr learned my addr 172.21.15.27:0/2962610024 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc19ffff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a80b6110 0x7fc1a8149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a77fe700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1a80b5c90 0x7fc1a8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 -- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1a80b5c90 msgr2=0x7fc1a8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1a80b5c90 0x7fc1a8158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 -- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a80b6110 msgr2=0x7fc1a8149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a80b6110 0x7fc1a8149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a7fff700 1 -- 172.21.15.27:0/2962610024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0009a70 con 0x7fc1a815fca0 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.440+0000 7fc1a77fe700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1a80b5c90 0x7fc1a8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.441+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 0x7fc1a80c6930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0014040 tx=0x7fc1a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.441+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0012430 con 0x7fc1a815fca0 2023-07-06T20:14:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.441+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a0020070 con 0x7fc1a815fca0 2023-07-06T20:14:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.441+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a8158200 con 0x7fc1a815fca0 2023-07-06T20:14:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.441+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0004280 con 0x7fc1a815fca0 2023-07-06T20:14:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.441+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a81587c0 con 0x7fc1a815fca0 2023-07-06T20:14:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.442+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a8158ce0 con 0x7fc1a815fca0 2023-07-06T20:14:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.443+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1a00043e0 con 0x7fc1a815fca0 2023-07-06T20:14:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.444+0000 7fc1a4ff9700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc18806ace0 0x7fc18806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.444+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc1a0022070 con 0x7fc1a815fca0 2023-07-06T20:14:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.445+0000 7fc1a77fe700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc18806ace0 0x7fc18806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.445+0000 7fc1a77fe700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc18806ace0 0x7fc18806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc198000f60 tx=0x7fc198007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.448+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc1a00bd310 con 0x7fc1a815fca0 2023-07-06T20:14:42.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.654+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1a80008d0 con 0x7fc1a815fca0 2023-07-06T20:14:42.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.655+0000 7fc1a4ff9700 1 -- 172.21.15.27:0/2962610024 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1a002c020 con 0x7fc1a815fca0 2023-07-06T20:14:42.657 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:42.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc18806ace0 msgr2=0x7fc18806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc18806ace0 0x7fc18806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc198000f60 tx=0x7fc198007d50 comp rx=0 tx=0).stop 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 msgr2=0x7fc1a80c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 0x7fc1a80c6930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0014040 tx=0x7fc1a0004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 shutdown_connections 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc18806ace0 0x7fc18806d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a815fca0 0x7fc1a80c6930 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc1a80b5c90 0x7fc1a8158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 --2- 172.21.15.27:0/2962610024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a80b6110 0x7fc1a8149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.657+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 >> 172.21.15.27:0/2962610024 conn(0x7fc1a806c150 msgr2=0x7fc1a815ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:42.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.658+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 shutdown_connections 2023-07-06T20:14:42.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.658+0000 7fc1adbff700 1 -- 172.21.15.27:0/2962610024 wait complete. 2023-07-06T20:14:42.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:42.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.882+0000 7fbea7bff700 1 Processor -- start 2023-07-06T20:14:42.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.882+0000 7fbea7bff700 1 -- start start 2023-07-06T20:14:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.883+0000 7fbea7bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbea015fca0 0x7fbea00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.883+0000 7fbea7bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea7bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea00cf780 con 0x7fbea00b60e0 2023-07-06T20:14:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea7bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea00cd490 con 0x7fbea015fca0 2023-07-06T20:14:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea7bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00cd770 con 0x7fbea00b5c90 2023-07-06T20:14:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbea015fca0 0x7fbea00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbea015fca0 0x7fbea00d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46382/0 (socket says 172.21.15.27:46382) 2023-07-06T20:14:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 -- 172.21.15.27:0/1128504466 learned_addr learned my addr 172.21.15.27:0/1128504466 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea519a700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 -- 172.21.15.27:0/1128504466 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 msgr2=0x7fbea00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 -- 172.21.15.27:0/1128504466 >> v1:172.21.15.153:6789/0 conn(0x7fbea00b60e0 legacy=0x7fbea00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea599b700 1 -- 172.21.15.27:0/1128504466 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00cda50 con 0x7fbea015fca0 2023-07-06T20:14:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.884+0000 7fbea519a700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea599b700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea00d1420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0098f0 tx=0x7fbe9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d488e647833fd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea4999700 1 -- 172.21.15.27:0/1128504466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c004230 con 0x7fbea015fca0 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea4999700 1 -- 172.21.15.27:0/1128504466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c004390 con 0x7fbea015fca0 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea4999700 1 -- 172.21.15.27:0/1128504466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0036a0 con 0x7fbea015fca0 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea7bff700 1 -- 172.21.15.27:0/1128504466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 msgr2=0x7fbea00d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea7bff700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea00d1420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0098f0 tx=0x7fbe9c009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea7bff700 1 -- 172.21.15.27:0/1128504466 shutdown_connections 2023-07-06T20:14:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea7bff700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea00d1420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.885+0000 7fbea7bff700 1 --2- 172.21.15.27:0/1128504466 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.886+0000 7fbea7bff700 1 -- 172.21.15.27:0/1128504466 >> 172.21.15.27:0/1128504466 conn(0x7fbea006c150 msgr2=0x7fbea006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.886+0000 7fbea7bff700 1 -- 172.21.15.27:0/1128504466 shutdown_connections 2023-07-06T20:14:42.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.886+0000 7fbea7bff700 1 -- 172.21.15.27:0/1128504466 wait complete. 2023-07-06T20:14:42.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.887+0000 7fbea7bff700 1 Processor -- start 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.888+0000 7fbea7bff700 1 -- start start 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea7bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea7bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbea00b60e0 0x7fbea015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea7bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea7bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00c9110 con 0x7fbea015fca0 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea7bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea015c1a0 con 0x7fbea00b60e0 2023-07-06T20:14:42.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea7bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea015c480 con 0x7fbea00b5c90 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea015d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46394/0 (socket says 172.21.15.27:46394) 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 -- 172.21.15.27:0/3978078844 learned_addr learned my addr 172.21.15.27:0/3978078844 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea599b700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea519a700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbea00b60e0 0x7fbea015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 -- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 msgr2=0x7fbea00c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 -- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbea00b60e0 msgr2=0x7fbea015b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbea00b60e0 0x7fbea015b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea619c700 1 -- 172.21.15.27:0/3978078844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c009550 con 0x7fbea015fca0 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.889+0000 7fbea519a700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbea00b60e0 0x7fbea015b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.890+0000 7fbea619c700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea015d240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800cba0 tx=0x7fbe9800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.890+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800aad0 con 0x7fbea015fca0 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.890+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9800ac30 con 0x7fbea015fca0 2023-07-06T20:14:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.890+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98012420 con 0x7fbea015fca0 2023-07-06T20:14:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.890+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea015c760 con 0x7fbea015fca0 2023-07-06T20:14:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.890+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea015cd20 con 0x7fbea015fca0 2023-07-06T20:14:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.891+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea00008d0 con 0x7fbea015fca0 2023-07-06T20:14:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.892+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe98012580 con 0x7fbea015fca0 2023-07-06T20:14:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.893+0000 7fbe967fc700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbe8406ace0 0x7fbe8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.893+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbe98089af0 con 0x7fbea015fca0 2023-07-06T20:14:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.894+0000 7fbea599b700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbe8406ace0 0x7fbe8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.894+0000 7fbea599b700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbe8406ace0 0x7fbe8406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009130 tx=0x7fbe9c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:42.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:42.897+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe98054bf0 con 0x7fbea015fca0 2023-07-06T20:14:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.107+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbea00d2470 con 0x7fbea015fca0 2023-07-06T20:14:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.107+0000 7fbe967fc700 1 -- 172.21.15.27:0/3978078844 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe98016020 con 0x7fbea015fca0 2023-07-06T20:14:43.109 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbe8406ace0 msgr2=0x7fbe8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbe8406ace0 0x7fbe8406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009130 tx=0x7fbe9c008030 comp rx=0 tx=0).stop 2023-07-06T20:14:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 msgr2=0x7fbea015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea015d240 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800cba0 tx=0x7fbe9800ceb0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 shutdown_connections 2023-07-06T20:14:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbe8406ace0 0x7fbe8406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015fca0 0x7fbea015d240 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbea00b5c90 0x7fbea00c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 --2- 172.21.15.27:0/3978078844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbea00b60e0 0x7fbea015b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 >> 172.21.15.27:0/3978078844 conn(0x7fbea006c150 msgr2=0x7fbea00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 shutdown_connections 2023-07-06T20:14:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.110+0000 7fbea7bff700 1 -- 172.21.15.27:0/3978078844 wait complete. 2023-07-06T20:14:43.129 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:43.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.331+0000 7f21d77ff700 1 Processor -- start 2023-07-06T20:14:43.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.331+0000 7f21d77ff700 1 -- start start 2023-07-06T20:14:43.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d77ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d0160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d77ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f21d015cbf0 0x7f21d015d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d77ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f21d00addb0 0x7f21d00ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d77ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00ae790 con 0x7f21d015cbf0 2023-07-06T20:14:43.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d77ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d01601f0 con 0x7f21d00addb0 2023-07-06T20:14:43.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d77ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d01607b0 con 0x7f21d015c820 2023-07-06T20:14:43.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d4d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f21d015cbf0 0x7f21d015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d4d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f21d015cbf0 0x7f21d015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46396/0 (socket says 172.21.15.27:46396) 2023-07-06T20:14:43.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.332+0000 7f21d4d9a700 1 -- 172.21.15.27:0/187468726 learned_addr learned my addr 172.21.15.27:0/187468726 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:43.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d559b700 1 --2- 172.21.15.27:0/187468726 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d0160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d5d9c700 1 --2- 172.21.15.27:0/187468726 >> v2:172.21.15.153:3300/0 conn(0x7f21d00addb0 0x7f21d00ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d4d9a700 1 -- 172.21.15.27:0/187468726 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 msgr2=0x7f21d0160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/187468726 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d0160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d4d9a700 1 -- 172.21.15.27:0/187468726 >> v2:172.21.15.153:3300/0 conn(0x7f21d00addb0 msgr2=0x7f21d00ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/187468726 >> v2:172.21.15.153:3300/0 conn(0x7f21d00addb0 0x7f21d00ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d4d9a700 1 -- 172.21.15.27:0/187468726 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d015d670 con 0x7f21d015cbf0 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/187468726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d015cbf0 0x7f21d015d060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f21d00b4ec0 tx=0x7f21c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6124fa92011dd83 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21c7fff700 1 -- 172.21.15.27:0/187468726 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000b870 con 0x7f21d015cbf0 2023-07-06T20:14:43.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21c7fff700 1 -- 172.21.15.27:0/187468726 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c000b9d0 con 0x7f21d015cbf0 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.333+0000 7f21c7fff700 1 -- 172.21.15.27:0/187468726 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000bd60 con 0x7f21d015cbf0 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 -- 172.21.15.27:0/187468726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d015cbf0 msgr2=0x7f21d015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 --2- 172.21.15.27:0/187468726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d015cbf0 0x7f21d015d060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f21d00b4ec0 tx=0x7f21c0009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 -- 172.21.15.27:0/187468726 shutdown_connections 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 --2- 172.21.15.27:0/187468726 >> v2:172.21.15.153:3300/0 conn(0x7f21d00addb0 0x7f21d00ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 --2- 172.21.15.27:0/187468726 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d0160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 --2- 172.21.15.27:0/187468726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d015cbf0 0x7f21d015d060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 -- 172.21.15.27:0/187468726 >> 172.21.15.27:0/187468726 conn(0x7f21d00cfbe0 msgr2=0x7f21d00b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:43.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 -- 172.21.15.27:0/187468726 shutdown_connections 2023-07-06T20:14:43.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.334+0000 7f21d77ff700 1 -- 172.21.15.27:0/187468726 wait complete. 2023-07-06T20:14:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.336+0000 7f21d77ff700 1 Processor -- start 2023-07-06T20:14:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.336+0000 7f21d77ff700 1 -- start start 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d77ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 0x7f21d01f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d77ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d00c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d77ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21d015cbf0 0x7f21d00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d77ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00b4670 con 0x7f21d00addb0 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d77ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00b48c0 con 0x7f21d015cbf0 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d77ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00b4ba0 con 0x7f21d015c820 2023-07-06T20:14:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d559b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 0x7f21d01f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d559b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 0x7f21d01f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46410/0 (socket says 172.21.15.27:46410) 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.337+0000 7f21d559b700 1 -- 172.21.15.27:0/3386776007 learned_addr learned my addr 172.21.15.27:0/3386776007 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d5d9c700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21d015cbf0 0x7f21d00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d559b700 1 -- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 msgr2=0x7f21d00c2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d00c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d559b700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d00c2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d559b700 1 -- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21d015cbf0 msgr2=0x7f21d00b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d559b700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21d015cbf0 0x7f21d00b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d559b700 1 -- 172.21.15.27:0/3386776007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009550 con 0x7f21d00addb0 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d5d9c700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21d015cbf0 0x7f21d00b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d00c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d559b700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 0x7f21d01f6400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e810 tx=0x7f21cc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc013610 con 0x7f21d00addb0 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc013770 con 0x7f21d00addb0 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc013b20 con 0x7f21d00addb0 2023-07-06T20:14:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.338+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d00b3180 con 0x7f21d00addb0 2023-07-06T20:14:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.339+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d00b2e00 con 0x7f21d00addb0 2023-07-06T20:14:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.339+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d0068270 con 0x7f21d00addb0 2023-07-06T20:14:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.341+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21cc013c80 con 0x7f21d00addb0 2023-07-06T20:14:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.341+0000 7f21c5ffb700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b406ace0 0x7f21b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.342+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f21cc089230 con 0x7f21d00addb0 2023-07-06T20:14:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.342+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b406ace0 0x7f21b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.343+0000 7f21d4d9a700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b406ace0 0x7f21b406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f21d00b4ec0 tx=0x7f21c0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:43.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.346+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21cc054210 con 0x7f21d00addb0 2023-07-06T20:14:43.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.552+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21d00008d0 con 0x7f21d00addb0 2023-07-06T20:14:43.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.553+0000 7f21c5ffb700 1 -- 172.21.15.27:0/3386776007 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f21cc057830 con 0x7f21d00addb0 2023-07-06T20:14:43.555 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:43.556 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:43.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b406ace0 msgr2=0x7f21b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b406ace0 0x7f21b406d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f21d00b4ec0 tx=0x7f21c0008030 comp rx=0 tx=0).stop 2023-07-06T20:14:43.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 msgr2=0x7f21d01f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 0x7f21d01f6400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00e810 tx=0x7f21cc00ebd0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 shutdown_connections 2023-07-06T20:14:43.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b406ace0 0x7f21b406d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21d015cbf0 0x7f21d00b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21d00addb0 0x7f21d01f6400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 --2- 172.21.15.27:0/3386776007 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21d015c820 0x7f21d00c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 >> 172.21.15.27:0/3386776007 conn(0x7f21d00cfbe0 msgr2=0x7f21d00b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:43.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 shutdown_connections 2023-07-06T20:14:43.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.556+0000 7f21d77ff700 1 -- 172.21.15.27:0/3386776007 wait complete. 2023-07-06T20:14:43.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:43.577 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.782+0000 7f0602dff700 1 Processor -- start 2023-07-06T20:14:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.782+0000 7f0602dff700 1 -- start start 2023-07-06T20:14:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.783+0000 7f0602dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.783+0000 7f0602dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f05fc0b6100 0x7f05fc0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.783+0000 7f0602dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0cf780 con 0x7f05fc0b5c90 2023-07-06T20:14:43.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.783+0000 7f0602dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc0cd490 con 0x7f05fc0b6100 2023-07-06T20:14:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.783+0000 7f0602dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cd770 con 0x7f05fc15fca0 2023-07-06T20:14:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f0600b9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f060139c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f05fc0b6100 0x7f05fc0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f0600b9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:57460/0 (socket says 172.21.15.27:57460) 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f0600b9b700 1 -- 172.21.15.27:0/3729650141 learned_addr learned my addr 172.21.15.27:0/3729650141 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f060139c700 1 -- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 msgr2=0x7f05fc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f060139c700 1 --2- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f060139c700 1 -- 172.21.15.27:0/3729650141 >> v1:172.21.15.153:6789/0 conn(0x7f05fc0b5c90 legacy=0x7f05fc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f060139c700 1 -- 172.21.15.27:0/3729650141 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0cda50 con 0x7f05fc0b6100 2023-07-06T20:14:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.784+0000 7f0600b9b700 1 --2- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.785+0000 7f060139c700 1 --2- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cfc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009d90 tx=0x7f05f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb71765c9e614fe1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.785+0000 7f05fb7fe700 1 -- 172.21.15.27:0/3729650141 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f400e070 con 0x7f05fc0b6100 2023-07-06T20:14:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.785+0000 7f05fb7fe700 1 -- 172.21.15.27:0/3729650141 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f400bb20 con 0x7f05fc0b6100 2023-07-06T20:14:43.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.785+0000 7f05fb7fe700 1 -- 172.21.15.27:0/3729650141 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4012430 con 0x7f05fc0b6100 2023-07-06T20:14:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.785+0000 7f0602dff700 1 -- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b6100 msgr2=0x7f05fc0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.785+0000 7f0602dff700 1 --2- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cfc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009d90 tx=0x7f05f400b330 comp rx=0 tx=0).stop 2023-07-06T20:14:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.786+0000 7f0602dff700 1 -- 172.21.15.27:0/3729650141 shutdown_connections 2023-07-06T20:14:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.786+0000 7f0602dff700 1 --2- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.786+0000 7f0602dff700 1 --2- 172.21.15.27:0/3729650141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cfc10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.786+0000 7f0602dff700 1 -- 172.21.15.27:0/3729650141 >> 172.21.15.27:0/3729650141 conn(0x7f05fc06c150 msgr2=0x7f05fc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.786+0000 7f0602dff700 1 -- 172.21.15.27:0/3729650141 shutdown_connections 2023-07-06T20:14:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.786+0000 7f0602dff700 1 -- 172.21.15.27:0/3729650141 wait complete. 2023-07-06T20:14:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.787+0000 7f0602dff700 1 Processor -- start 2023-07-06T20:14:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.788+0000 7f0602dff700 1 -- start start 2023-07-06T20:14:43.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.788+0000 7f0602dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b5c90 0x7f05fc15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.788+0000 7f0602dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.788+0000 7f0602dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.788+0000 7f0602dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc15ddf0 con 0x7f05fc0b6100 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f0602dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc15e930 con 0x7f05fc0b5c90 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f0602dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc15e000 con 0x7f05fc15fca0 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f0600b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b5c90 0x7f05fc15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f05fbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f0600b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b5c90 0x7f05fc15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46780/0 (socket says 172.21.15.27:46780) 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f0600b9b700 1 -- 172.21.15.27:0/2525051762 learned_addr learned my addr 172.21.15.27:0/2525051762 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f060139c700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f05fbfff700 1 -- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 msgr2=0x7f05fc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f05fbfff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f05fbfff700 1 -- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b5c90 msgr2=0x7f05fc15bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f05fbfff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b5c90 0x7f05fc15bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.789+0000 7f05fbfff700 1 -- 172.21.15.27:0/2525051762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0009550 con 0x7f05fc0b6100 2023-07-06T20:14:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.790+0000 7f05fbfff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cc630 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000f60 tx=0x7f05e800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.790+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e80052d0 con 0x7f05fc0b6100 2023-07-06T20:14:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.790+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8005430 con 0x7f05fc0b6100 2023-07-06T20:14:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.790+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f4009a70 con 0x7f05fc0b6100 2023-07-06T20:14:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.790+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e80038c0 con 0x7f05fc0b6100 2023-07-06T20:14:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.790+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc15e5c0 con 0x7f05fc0b6100 2023-07-06T20:14:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.791+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc15eaa0 con 0x7f05fc0b6100 2023-07-06T20:14:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.792+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f05e8003a20 con 0x7f05fc0b6100 2023-07-06T20:14:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.793+0000 7f05f97fa700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05e006ace0 0x7f05e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.793+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f05e8088900 con 0x7f05fc0b6100 2023-07-06T20:14:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.794+0000 7f0600b9b700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05e006ace0 0x7f05e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.794+0000 7f0600b9b700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05e006ace0 0x7f05e006d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d80 tx=0x7f05f00092c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:43.797+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f05e80538d0 con 0x7f05fc0b6100 2023-07-06T20:14:44.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.005+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05fc0008d0 con 0x7f05fc0b6100 2023-07-06T20:14:44.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.005+0000 7f05f97fa700 1 -- 172.21.15.27:0/2525051762 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05e8017020 con 0x7f05fc0b6100 2023-07-06T20:14:44.007 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05e006ace0 msgr2=0x7f05e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05e006ace0 0x7f05e006d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d80 tx=0x7f05f00092c0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05fc0b6100 msgr2=0x7f05fc0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cc630 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000f60 tx=0x7f05e800c730 comp rx=0 tx=0).stop 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 shutdown_connections 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05e006ace0 0x7f05e006d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05fc15fca0 0x7f05fc0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05fc0b5c90 0x7f05fc15bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 --2- 172.21.15.27:0/2525051762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05fc0b6100 0x7f05fc0cc630 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.007+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 >> 172.21.15.27:0/2525051762 conn(0x7f05fc06c150 msgr2=0x7f05fc0b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.008+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 shutdown_connections 2023-07-06T20:14:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.008+0000 7f0602dff700 1 -- 172.21.15.27:0/2525051762 wait complete. 2023-07-06T20:14:44.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:44.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.232+0000 7f86d5dff700 1 Processor -- start 2023-07-06T20:14:44.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.232+0000 7f86d5dff700 1 -- start start 2023-07-06T20:14:44.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.233+0000 7f86d5dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86d015fca0 0x7f86d00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.233+0000 7f86d5dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 0x7f86d00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86d5dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d00cf780 con 0x7f86d00b6110 2023-07-06T20:14:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86d5dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00cd490 con 0x7f86d00b5c90 2023-07-06T20:14:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86d5dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00cd770 con 0x7f86d015fca0 2023-07-06T20:14:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86ceffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 0x7f86d00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86cf7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86d015fca0 0x7f86d00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86ceffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 0x7f86d00d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46418/0 (socket says 172.21.15.27:46418) 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86ceffd700 1 -- 172.21.15.27:0/3172396502 learned_addr learned my addr 172.21.15.27:0/3172396502 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86cf7fe700 1 -- 172.21.15.27:0/3172396502 >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 msgr2=0x7f86d00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86cf7fe700 1 --2- 172.21.15.27:0/3172396502 >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 0x7f86d00d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86cf7fe700 1 -- 172.21.15.27:0/3172396502 >> v1:172.21.15.153:6789/0 conn(0x7f86d00b6110 legacy=0x7f86d00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86cf7fe700 1 -- 172.21.15.27:0/3172396502 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d00cda50 con 0x7f86d015fca0 2023-07-06T20:14:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.234+0000 7f86ceffd700 1 --2- 172.21.15.27:0/3172396502 >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 0x7f86d00d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.235+0000 7f86cf7fe700 1 --2- 172.21.15.27:0/3172396502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d015fca0 0x7f86d00d1420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86d00d0930 tx=0x7f86c0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91c78b1f400b41c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.235+0000 7f86ce7fc700 1 -- 172.21.15.27:0/3172396502 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0011830 con 0x7f86d015fca0 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.235+0000 7f86ce7fc700 1 -- 172.21.15.27:0/3172396502 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0011990 con 0x7f86d015fca0 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.235+0000 7f86ce7fc700 1 -- 172.21.15.27:0/3172396502 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0011d20 con 0x7f86d015fca0 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.235+0000 7f86d5dff700 1 -- 172.21.15.27:0/3172396502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d015fca0 msgr2=0x7f86d00d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.235+0000 7f86d5dff700 1 --2- 172.21.15.27:0/3172396502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d015fca0 0x7f86d00d1420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86d00d0930 tx=0x7f86c0009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.236+0000 7f86d5dff700 1 -- 172.21.15.27:0/3172396502 shutdown_connections 2023-07-06T20:14:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.236+0000 7f86d5dff700 1 --2- 172.21.15.27:0/3172396502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d015fca0 0x7f86d00d1420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.236+0000 7f86d5dff700 1 --2- 172.21.15.27:0/3172396502 >> v2:172.21.15.27:3300/0 conn(0x7f86d00b5c90 0x7f86d00d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.236+0000 7f86d5dff700 1 -- 172.21.15.27:0/3172396502 >> 172.21.15.27:0/3172396502 conn(0x7f86d006c150 msgr2=0x7f86d006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.236+0000 7f86d5dff700 1 -- 172.21.15.27:0/3172396502 shutdown_connections 2023-07-06T20:14:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.236+0000 7f86d5dff700 1 -- 172.21.15.27:0/3172396502 wait complete. 2023-07-06T20:14:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.238+0000 7f86d5dff700 1 Processor -- start 2023-07-06T20:14:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.238+0000 7f86d5dff700 1 -- start start 2023-07-06T20:14:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86d5dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d00b5c90 0x7f86d00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86d5dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86d00b6110 0x7f86d00c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86d5dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 0x7f86d00caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86d5dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01f2ec0 con 0x7f86d015fca0 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86d5dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00ccca0 con 0x7f86d00b5c90 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86d5dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00cc120 con 0x7f86d00b6110 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 0x7f86d00caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 0x7f86d00caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46420/0 (socket says 172.21.15.27:46420) 2023-07-06T20:14:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 -- 172.21.15.27:0/1043955283 learned_addr learned my addr 172.21.15.27:0/1043955283 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cf7fe700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d00b5c90 0x7f86d00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86ceffd700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86d00b6110 0x7f86d00c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 -- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86d00b6110 msgr2=0x7f86d00c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86d00b6110 0x7f86d00c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 -- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d00b5c90 msgr2=0x7f86d00cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d00b5c90 0x7f86d00cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.240+0000 7f86cffff700 1 -- 172.21.15.27:0/1043955283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4009550 con 0x7f86d015fca0 2023-07-06T20:14:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.241+0000 7f86cffff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 0x7f86d00caa40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cc50 tx=0x7f86c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:44.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.241+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8003c00 con 0x7f86d015fca0 2023-07-06T20:14:44.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.241+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c8003d60 con 0x7f86d015fca0 2023-07-06T20:14:44.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.241+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800aab0 con 0x7f86d015fca0 2023-07-06T20:14:44.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.241+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c00094a0 con 0x7f86d015fca0 2023-07-06T20:14:44.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.241+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d00cc6e0 con 0x7f86d015fca0 2023-07-06T20:14:44.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.242+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d00c6910 con 0x7f86d015fca0 2023-07-06T20:14:44.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.243+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86c800ac10 con 0x7f86d015fca0 2023-07-06T20:14:44.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.244+0000 7f86b7fff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f86bc06ace0 0x7f86bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.245+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f86c80894a0 con 0x7f86d015fca0 2023-07-06T20:14:44.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.245+0000 7f86cf7fe700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f86bc06ace0 0x7f86bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.246+0000 7f86cf7fe700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f86bc06ace0 0x7f86bc06d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f86c00090b0 tx=0x7f86c001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.249+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86d00c6910 con 0x7f86d015fca0 2023-07-06T20:14:44.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.455+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f86d00c6910 con 0x7f86d015fca0 2023-07-06T20:14:44.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.455+0000 7f86b7fff700 1 -- 172.21.15.27:0/1043955283 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f86d00c6910 con 0x7f86d015fca0 2023-07-06T20:14:44.457 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:44.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f86bc06ace0 msgr2=0x7f86bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f86bc06ace0 0x7f86bc06d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f86c00090b0 tx=0x7f86c001c040 comp rx=0 tx=0).stop 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 msgr2=0x7f86d00caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 0x7f86d00caa40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cc50 tx=0x7f86c800cf60 comp rx=0 tx=0).stop 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 shutdown_connections 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f86bc06ace0 0x7f86bc06d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86d015fca0 0x7f86d00caa40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d00b5c90 0x7f86d00cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 --2- 172.21.15.27:0/1043955283 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86d00b6110 0x7f86d00c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.457+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 >> 172.21.15.27:0/1043955283 conn(0x7f86d006c150 msgr2=0x7f86d00b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.458+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 shutdown_connections 2023-07-06T20:14:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.458+0000 7f86d5dff700 1 -- 172.21.15.27:0/1043955283 wait complete. 2023-07-06T20:14:44.478 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.681+0000 7f2314bff700 1 Processor -- start 2023-07-06T20:14:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.681+0000 7f2314bff700 1 -- start start 2023-07-06T20:14:44.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2314bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23100b5c90 0x7f23100d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2314bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f23100b6110 0x7f23100d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2314bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23100cf780 con 0x7f231015fca0 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2314bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23100cd490 con 0x7f23100b5c90 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2314bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23100cd770 con 0x7f23100b6110 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2307fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23100b5c90 0x7f23100d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f230e59c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f231015fca0 0x7f23100d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56702/0 (socket says 172.21.15.27:56702) 2023-07-06T20:14:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f230e59c700 1 -- 172.21.15.27:0/2925921030 learned_addr learned my addr 172.21.15.27:0/2925921030 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f230ed9d700 1 --2- 172.21.15.27:0/2925921030 >> v2:172.21.15.153:3300/0 conn(0x7f23100b6110 0x7f23100d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2307fff700 1 -- 172.21.15.27:0/2925921030 >> v2:172.21.15.153:3300/0 conn(0x7f23100b6110 msgr2=0x7f23100d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2307fff700 1 --2- 172.21.15.27:0/2925921030 >> v2:172.21.15.153:3300/0 conn(0x7f23100b6110 0x7f23100d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2307fff700 1 -- 172.21.15.27:0/2925921030 >> v1:172.21.15.27:6789/0 conn(0x7f231015fca0 legacy=0x7f23100d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f2307fff700 1 -- 172.21.15.27:0/2925921030 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23100cda50 con 0x7f23100b5c90 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.683+0000 7f230dd9b700 1 -- 172.21.15.27:0/2925921030 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299258712 0 0) 0x7f23100cf780 con 0x7f231015fca0 2023-07-06T20:14:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.684+0000 7f2307fff700 1 --2- 172.21.15.27:0/2925921030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100d2d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23100cd410 tx=0x7f22f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f83bc6de5bdda1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.684+0000 7f230dd9b700 1 -- 172.21.15.27:0/2925921030 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8011830 con 0x7f23100b5c90 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.684+0000 7f230dd9b700 1 -- 172.21.15.27:0/2925921030 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f8011990 con 0x7f23100b5c90 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.684+0000 7f230dd9b700 1 -- 172.21.15.27:0/2925921030 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8011d20 con 0x7f23100b5c90 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.684+0000 7f2314bff700 1 -- 172.21.15.27:0/2925921030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 msgr2=0x7f23100d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.684+0000 7f2314bff700 1 --2- 172.21.15.27:0/2925921030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100d2d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23100cd410 tx=0x7f22f8009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.685+0000 7f2314bff700 1 -- 172.21.15.27:0/2925921030 shutdown_connections 2023-07-06T20:14:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.685+0000 7f2314bff700 1 --2- 172.21.15.27:0/2925921030 >> v2:172.21.15.153:3300/0 conn(0x7f23100b6110 0x7f23100d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.685+0000 7f2314bff700 1 --2- 172.21.15.27:0/2925921030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100d2d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.685+0000 7f2314bff700 1 -- 172.21.15.27:0/2925921030 >> 172.21.15.27:0/2925921030 conn(0x7f231006c150 msgr2=0x7f231006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.685+0000 7f2314bff700 1 -- 172.21.15.27:0/2925921030 shutdown_connections 2023-07-06T20:14:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.685+0000 7f2314bff700 1 -- 172.21.15.27:0/2925921030 wait complete. 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.686+0000 7f2314bff700 1 Processor -- start 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.687+0000 7f2314bff700 1 -- start start 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2314bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2314bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23100b6110 0x7f23100d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2314bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f231015fca0 0x7f23100d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2314bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c7730 con 0x7f23100b5c90 2023-07-06T20:14:44.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2314bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c6ab0 con 0x7f23100b6110 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2314bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c6d90 con 0x7f231015fca0 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46438/0 (socket says 172.21.15.27:46438) 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 -- 172.21.15.27:0/2842318121 learned_addr learned my addr 172.21.15.27:0/2842318121 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f2307fff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23100b6110 0x7f23100d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230ed9d700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f231015fca0 0x7f23100d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 -- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f231015fca0 msgr2=0x7f23100d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f231015fca0 0x7f23100d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 -- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23100b6110 msgr2=0x7f23100d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23100b6110 0x7f23100d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.688+0000 7f230e59c700 1 -- 172.21.15.27:0/2842318121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f80094a0 con 0x7f23100b5c90 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.689+0000 7f230e59c700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100cc410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f230000c550 tx=0x7f230000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.689+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300004c80 con 0x7f23100b5c90 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.689+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300004e00 con 0x7f23100b5c90 2023-07-06T20:14:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.689+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23100c7070 con 0x7f23100b5c90 2023-07-06T20:14:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.689+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300007310 con 0x7f23100b5c90 2023-07-06T20:14:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.689+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23101f2020 con 0x7f23100b5c90 2023-07-06T20:14:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.690+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23100008d0 con 0x7f23100b5c90 2023-07-06T20:14:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.691+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2300007470 con 0x7f23100b5c90 2023-07-06T20:14:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.693+0000 7f2306ffd700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f22f006ace0 0x7f22f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.693+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2300089120 con 0x7f23100b5c90 2023-07-06T20:14:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.693+0000 7f2307fff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f22f006ace0 0x7f22f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.694+0000 7f2307fff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f22f006ace0 0x7f22f006d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22f80090b0 tx=0x7f22f8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:44.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.697+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23000540f0 con 0x7f23100b5c90 2023-07-06T20:14:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.907+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23100cfdf0 con 0x7f23100b5c90 2023-07-06T20:14:44.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.907+0000 7f2306ffd700 1 -- 172.21.15.27:0/2842318121 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2300057710 con 0x7f23100b5c90 2023-07-06T20:14:44.909 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:44.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f22f006ace0 msgr2=0x7f22f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f22f006ace0 0x7f22f006d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22f80090b0 tx=0x7f22f8008030 comp rx=0 tx=0).stop 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 msgr2=0x7f23100cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100cc410 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f230000c550 tx=0x7f230000c860 comp rx=0 tx=0).stop 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 shutdown_connections 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f22f006ace0 0x7f22f006d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f231015fca0 0x7f23100d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23100b5c90 0x7f23100cc410 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 --2- 172.21.15.27:0/2842318121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23100b6110 0x7f23100d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:44.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 >> 172.21.15.27:0/2842318121 conn(0x7f231006c150 msgr2=0x7f23100d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:44.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 shutdown_connections 2023-07-06T20:14:44.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:44.910+0000 7f2314bff700 1 -- 172.21.15.27:0/2842318121 wait complete. 2023-07-06T20:14:44.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.136+0000 7f5ed5fff700 1 Processor -- start 2023-07-06T20:14:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.136+0000 7f5ed5fff700 1 -- start start 2023-07-06T20:14:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.137+0000 7f5ed5fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ed00b8610 0x7f5ed00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.137+0000 7f5ed5fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00ceed0 0x7f5ed00cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.137+0000 7f5ed5fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00d1370 con 0x7f5ed00cfcb0 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.137+0000 7f5ed5fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed00d1650 con 0x7f5ed00b8610 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.137+0000 7f5ed5fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00d0100 con 0x7f5ed00ceed0 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.137+0000 7f5ecf7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ed00b8610 0x7f5ed00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ed00b8610 0x7f5ed00b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46446/0 (socket says 172.21.15.27:46446) 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2175539972 learned_addr learned my addr 172.21.15.27:0/2175539972 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5eceffd700 1 --2- 172.21.15.27:0/2175539972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00ceed0 0x7f5ed00cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2175539972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00ceed0 msgr2=0x7f5ed00cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 --2- 172.21.15.27:0/2175539972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00ceed0 0x7f5ed00cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2175539972 >> v1:172.21.15.153:6789/0 conn(0x7f5ed00cfcb0 legacy=0x7f5ed0160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2175539972 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed00b8cd0 con 0x7f5ed00b8610 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.138+0000 7f5ecf7fe700 1 --2- 172.21.15.27:0/2175539972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed00b2180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098f0 tx=0x7f5ec0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a21e5c4a6989ea server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ece7fc700 1 -- 172.21.15.27:0/2175539972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000b8a0 con 0x7f5ed00b8610 2023-07-06T20:14:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ece7fc700 1 -- 172.21.15.27:0/2175539972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000ba00 con 0x7f5ed00b8610 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ece7fc700 1 -- 172.21.15.27:0/2175539972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000bd90 con 0x7f5ed00b8610 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2175539972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 msgr2=0x7f5ed00b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2175539972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed00b2180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098f0 tx=0x7f5ec0009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2175539972 shutdown_connections 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2175539972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed00b2180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2175539972 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00ceed0 0x7f5ed00cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2175539972 >> 172.21.15.27:0/2175539972 conn(0x7f5ed0071690 msgr2=0x7f5ed0071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2175539972 shutdown_connections 2023-07-06T20:14:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.139+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2175539972 wait complete. 2023-07-06T20:14:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.142+0000 7f5ed5fff700 1 Processor -- start 2023-07-06T20:14:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.142+0000 7f5ed5fff700 1 -- start start 2023-07-06T20:14:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.143+0000 7f5ed5fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.143+0000 7f5ed5fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ed00ceed0 0x7f5ed01f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.143+0000 7f5ed5fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00cfcb0 0x7f5ed01f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.143+0000 7f5ed5fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed015c3b0 con 0x7f5ed00b8610 2023-07-06T20:14:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.143+0000 7f5ed5fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00cc800 con 0x7f5ed00ceed0 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.143+0000 7f5ed5fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00cb910 con 0x7f5ed00cfcb0 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46458/0 (socket says 172.21.15.27:46458) 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2484968722 learned_addr learned my addr 172.21.15.27:0/2484968722 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecffff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00cfcb0 0x7f5ed01f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5eceffd700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ed00ceed0 0x7f5ed01f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00cfcb0 msgr2=0x7f5ed01f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00cfcb0 0x7f5ed01f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ed00ceed0 msgr2=0x7f5ed01f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ed00ceed0 0x7f5ed01f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 -- 172.21.15.27:0/2484968722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009550 con 0x7f5ed00b8610 2023-07-06T20:14:45.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.144+0000 7f5ecf7fe700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed015be70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009c40 tx=0x7f5ec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.145+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004510 con 0x7f5ed00b8610 2023-07-06T20:14:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.145+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec001e070 con 0x7f5ed00b8610 2023-07-06T20:14:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.145+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed00cbbf0 con 0x7f5ed00b8610 2023-07-06T20:14:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.145+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00038e0 con 0x7f5ed00b8610 2023-07-06T20:14:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.145+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed00cc1b0 con 0x7f5ed00b8610 2023-07-06T20:14:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.146+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed00008d0 con 0x7f5ed00b8610 2023-07-06T20:14:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.147+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ec0003a40 con 0x7f5ed00b8610 2023-07-06T20:14:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.148+0000 7f5eb7fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ebc06ace0 0x7f5ebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.148+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5ec0020070 con 0x7f5ed00b8610 2023-07-06T20:14:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.148+0000 7f5eceffd700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ebc06ace0 0x7f5ebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.149+0000 7f5eceffd700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ebc06ace0 0x7f5ebc06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00b43f0 tx=0x7f5ec4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:45.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.152+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5ec00559f0 con 0x7f5ed00b8610 2023-07-06T20:14:45.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.361+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ed00cd550 con 0x7f5ed00b8610 2023-07-06T20:14:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.362+0000 7f5eb7fff700 1 -- 172.21.15.27:0/2484968722 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ec0059010 con 0x7f5ed00b8610 2023-07-06T20:14:45.364 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ebc06ace0 msgr2=0x7f5ebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ebc06ace0 0x7f5ebc06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00b43f0 tx=0x7f5ec4008040 comp rx=0 tx=0).stop 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 msgr2=0x7f5ed015be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed015be70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009c40 tx=0x7f5ec0004750 comp rx=0 tx=0).stop 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 shutdown_connections 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ebc06ace0 0x7f5ebc06d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5ed00cfcb0 0x7f5ed01f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ed00b8610 0x7f5ed015be70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 --2- 172.21.15.27:0/2484968722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ed00ceed0 0x7f5ed01f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.364+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 >> 172.21.15.27:0/2484968722 conn(0x7f5ed0071690 msgr2=0x7f5ed015ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.365+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 shutdown_connections 2023-07-06T20:14:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.365+0000 7f5ed5fff700 1 -- 172.21.15.27:0/2484968722 wait complete. 2023-07-06T20:14:45.389 INFO:tasks.cephfs_test_runner:Starting test: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-07-06T20:14:45.390 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-07-06T20:14:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.592+0000 7f3aab7ff700 1 Processor -- start 2023-07-06T20:14:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.592+0000 7f3aab7ff700 1 -- start start 2023-07-06T20:14:45.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.593+0000 7f3aab7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3aa40b6110 0x7f3aa40cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.593+0000 7f3aab7ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa40cf780 con 0x7f3aa40b5c90 2023-07-06T20:14:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aab7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa40cd490 con 0x7f3aa415fca0 2023-07-06T20:14:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aab7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa40cd770 con 0x7f3aa40b6110 2023-07-06T20:14:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aa9d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3aa40b6110 0x7f3aa40cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aa8d9a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3aa40b5c90 0x7f3aa40d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56706/0 (socket says 172.21.15.27:56706) 2023-07-06T20:14:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aa8d9a700 1 -- 172.21.15.27:0/2091000685 learned_addr learned my addr 172.21.15.27:0/2091000685 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/2091000685 >> v1:172.21.15.153:6789/0 conn(0x7f3aa415fca0 legacy=0x7f3aa40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/2091000685 >> v1:172.21.15.27:6789/0 conn(0x7f3aa40b5c90 legacy=0x7f3aa40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.594+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/2091000685 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40cda50 con 0x7f3aa40b6110 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3aa9d9c700 1 --2- 172.21.15.27:0/2091000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa40b6110 0x7f3aa40cfc10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009d90 tx=0x7f3a9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a7f702b7f4f931 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3a9bfff700 1 -- 172.21.15.27:0/2091000685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00e070 con 0x7f3aa40b6110 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3a9bfff700 1 -- 172.21.15.27:0/2091000685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c00bbd0 con 0x7f3aa40b6110 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3a9bfff700 1 -- 172.21.15.27:0/2091000685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c012430 con 0x7f3aa40b6110 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3aab7ff700 1 -- 172.21.15.27:0/2091000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa40b6110 msgr2=0x7f3aa40cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/2091000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa40b6110 0x7f3aa40cfc10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009d90 tx=0x7f3a9c00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:45.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3aab7ff700 1 -- 172.21.15.27:0/2091000685 shutdown_connections 2023-07-06T20:14:45.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/2091000685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa40b6110 0x7f3aa40cfc10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.595+0000 7f3aab7ff700 1 -- 172.21.15.27:0/2091000685 >> 172.21.15.27:0/2091000685 conn(0x7f3aa406c150 msgr2=0x7f3aa406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:45.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.596+0000 7f3aab7ff700 1 -- 172.21.15.27:0/2091000685 shutdown_connections 2023-07-06T20:14:45.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.596+0000 7f3aab7ff700 1 -- 172.21.15.27:0/2091000685 wait complete. 2023-07-06T20:14:45.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.597+0000 7f3aab7ff700 1 Processor -- start 2023-07-06T20:14:45.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.598+0000 7f3aab7ff700 1 -- start start 2023-07-06T20:14:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.598+0000 7f3aab7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40b5c90 0x7f3aa41f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aab7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40b6110 0x7f3aa41576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aab7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 0x7f3aa40d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aab7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41f62a0 con 0x7f3aa415fca0 2023-07-06T20:14:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aab7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4157c10 con 0x7f3aa40b6110 2023-07-06T20:14:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aab7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41f2070 con 0x7f3aa40b5c90 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 0x7f3aa40d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 0x7f3aa40d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46480/0 (socket says 172.21.15.27:46480) 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/21953573 learned_addr learned my addr 172.21.15.27:0/21953573 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa959b700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40b5c90 0x7f3aa41f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa8d9a700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40b6110 0x7f3aa41576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40b5c90 msgr2=0x7f3aa41f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40b5c90 0x7f3aa41f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40b6110 msgr2=0x7f3aa41576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40b6110 0x7f3aa41576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.599+0000 7f3aa9d9c700 1 -- 172.21.15.27:0/21953573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c009a70 con 0x7f3aa415fca0 2023-07-06T20:14:45.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.600+0000 7f3aa9d9c700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 0x7f3aa40d2b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c014040 tx=0x7f3a9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:45.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.600+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c012d20 con 0x7f3aa415fca0 2023-07-06T20:14:45.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.600+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c020070 con 0x7f3aa415fca0 2023-07-06T20:14:45.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.600+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c012ea0 con 0x7f3aa415fca0 2023-07-06T20:14:45.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.600+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa41f2350 con 0x7f3aa415fca0 2023-07-06T20:14:45.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.600+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa41f2910 con 0x7f3aa415fca0 2023-07-06T20:14:45.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.601+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa40008d0 con 0x7f3aa415fca0 2023-07-06T20:14:45.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.602+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a9c004280 con 0x7f3aa415fca0 2023-07-06T20:14:45.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.603+0000 7f3a99ffb700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3a8806ace0 0x7f3a8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:45.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.603+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3a9c022070 con 0x7f3aa415fca0 2023-07-06T20:14:45.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.603+0000 7f3aa959b700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3a8806ace0 0x7f3a8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:45.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.604+0000 7f3aa959b700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3a8806ace0 0x7f3a8806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0007fd0 tx=0x7f3aa0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:45.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.607+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a9c0562d0 con 0x7f3aa415fca0 2023-07-06T20:14:45.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.896+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f3aa40cfe90 con 0x7f3aa415fca0 2023-07-06T20:14:45.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.995+0000 7f3a99ffb700 1 -- 172.21.15.27:0/21953573 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v70) v1 ==== 163+0+0 (secure 0 0 0) 0x7f3a9c0598f0 con 0x7f3aa415fca0 2023-07-06T20:14:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3a8806ace0 msgr2=0x7f3a8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3a8806ace0 0x7f3a8806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0007fd0 tx=0x7f3aa0009040 comp rx=0 tx=0).stop 2023-07-06T20:14:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 msgr2=0x7f3aa40d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 0x7f3aa40d2b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c014040 tx=0x7f3a9c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 shutdown_connections 2023-07-06T20:14:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3a8806ace0 0x7f3a8806d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aa415fca0 0x7f3aa40d2b60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3aa40b5c90 0x7f3aa41f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 --2- 172.21.15.27:0/21953573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa40b6110 0x7f3aa41576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 >> 172.21.15.27:0/21953573 conn(0x7f3aa406c150 msgr2=0x7f3aa415aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 shutdown_connections 2023-07-06T20:14:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:45.998+0000 7f3aab7ff700 1 -- 172.21.15.27:0/21953573 wait complete. 2023-07-06T20:14:46.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T20:14:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.223+0000 7f5053fff700 1 Processor -- start 2023-07-06T20:14:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.223+0000 7f5053fff700 1 -- start start 2023-07-06T20:14:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.224+0000 7f5053fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f504c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.224+0000 7f5053fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c0cf780 con 0x7f504c15fca0 2023-07-06T20:14:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.224+0000 7f5053fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c0cd490 con 0x7f504c0b5c90 2023-07-06T20:14:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.224+0000 7f5053fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0cd770 con 0x7f504c0b6110 2023-07-06T20:14:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.224+0000 7f5051d9b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f504c15fca0 0x7f504c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56718/0 (socket says 172.21.15.27:56718) 2023-07-06T20:14:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.224+0000 7f5051d9b700 1 -- 172.21.15.27:0/3045279119 learned_addr learned my addr 172.21.15.27:0/3045279119 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f505259c700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f504c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930879361 0 0) 0x7f504c0cf780 con 0x7f504c15fca0 2023-07-06T20:14:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50380035b0 con 0x7f504c15fca0 2023-07-06T20:14:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3181602957 0 0) 0x7f50380035b0 con 0x7f504c15fca0 2023-07-06T20:14:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 msgr2=0x7f504c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f504c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 >> v1:172.21.15.153:6789/0 conn(0x7f504c0b5c90 legacy=0x7f504c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0cda50 con 0x7f504c15fca0 2023-07-06T20:14:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f5048001670 con 0x7f504c15fca0 2023-07-06T20:14:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 >> v1:172.21.15.27:6789/0 conn(0x7f504c15fca0 legacy=0x7f504c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5038003a30 0x7f5038003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f5038004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f50380050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0cd770 con 0x7f50380043d0 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503800ba10 con 0x7f5038003a30 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.225+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503800bba0 con 0x7f504c0b6110 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f5038004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f5051d9b700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5038003a30 0x7f5038003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505259c700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f50380050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 -- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 msgr2=0x7f50380050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f50380050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 -- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5038003a30 msgr2=0x7f5038003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5038003a30 0x7f5038003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 -- 172.21.15.27:0/3045279119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380035b0 con 0x7f50380043d0 2023-07-06T20:14:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505259c700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c0b6110 0x7f50380050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.226+0000 7f505159a700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f5038004890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f503c0018b0 tx=0x7f5038005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd744ff7888e37b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c001370 con 0x7f50380043d0 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c00c6b0 con 0x7f50380043d0 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5050d99700 1 -- 172.21.15.27:0/3045279119 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00ca60 con 0x7f50380043d0 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5053fff700 1 -- 172.21.15.27:0/3045279119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 msgr2=0x7f5038004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5053fff700 1 --2- 172.21.15.27:0/3045279119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f5038004890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f503c0018b0 tx=0x7f5038005880 comp rx=0 tx=0).stop 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f505259c700 1 -- 172.21.15.27:0/3045279119 reap_dead start 2023-07-06T20:14:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5053fff700 1 -- 172.21.15.27:0/3045279119 shutdown_connections 2023-07-06T20:14:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5053fff700 1 -- 172.21.15.27:0/3045279119 >> 172.21.15.27:0/3045279119 conn(0x7f504c06c150 msgr2=0x7f504c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5053fff700 1 -- 172.21.15.27:0/3045279119 shutdown_connections 2023-07-06T20:14:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.227+0000 7f5053fff700 1 -- 172.21.15.27:0/3045279119 wait complete. 2023-07-06T20:14:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.229+0000 7f5053fff700 1 Processor -- start 2023-07-06T20:14:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.229+0000 7f5053fff700 1 -- start start 2023-07-06T20:14:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5053fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f504c15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5053fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c148a50 0x7f504c14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5053fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f504c14afe0 0x7f504c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5053fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c14ba80 con 0x7f50380043d0 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5053fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c15e060 con 0x7f504c14afe0 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5053fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c15da50 con 0x7f504c148a50 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5051d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f504c15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5051d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f504c15c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46506/0 (socket says 172.21.15.27:46506) 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.230+0000 7f5051d9b700 1 -- 172.21.15.27:0/3332297836 learned_addr learned my addr 172.21.15.27:0/3332297836 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f505259c700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f504c14afe0 0x7f504c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5051d9b700 1 -- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c148a50 msgr2=0x7f504c14aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5051d9b700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c148a50 0x7f504c14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5051d9b700 1 -- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f504c14afe0 msgr2=0x7f504c0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5051d9b700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f504c14afe0 0x7f504c0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5051d9b700 1 -- 172.21.15.27:0/3332297836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c14b4c0 con 0x7f50380043d0 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f505259c700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f504c14afe0 0x7f504c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5051d9b700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f504c15c020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5048007290 tx=0x7f50480075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480077e0 con 0x7f50380043d0 2023-07-06T20:14:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504800f7f0 con 0x7f50380043d0 2023-07-06T20:14:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504800fba0 con 0x7f50380043d0 2023-07-06T20:14:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.231+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c14b7a0 con 0x7f50380043d0 2023-07-06T20:14:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.232+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c15c560 con 0x7f50380043d0 2023-07-06T20:14:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.233+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c0d27d0 con 0x7f50380043d0 2023-07-06T20:14:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.233+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f504800fd00 con 0x7f50380043d0 2023-07-06T20:14:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.235+0000 7f50427fc700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503006ace0 0x7f503006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.235+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5048089580 con 0x7f50380043d0 2023-07-06T20:14:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.236+0000 7f505159a700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503006ace0 0x7f503006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.237+0000 7f505159a700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503006ace0 0x7f503006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f503c0018b0 tx=0x7f503c007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.240+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5048057fe0 con 0x7f50380043d0 2023-07-06T20:14:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.451+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f504c0008d0 con 0x7f50380043d0 2023-07-06T20:14:46.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.456+0000 7f50427fc700 1 -- 172.21.15.27:0/3332297836 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5048057b70 con 0x7f50380043d0 2023-07-06T20:14:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503006ace0 msgr2=0x7f503006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503006ace0 0x7f503006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f503c0018b0 tx=0x7f503c007f90 comp rx=0 tx=0).stop 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 msgr2=0x7f504c15c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f504c15c020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5048007290 tx=0x7f50480075a0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 shutdown_connections 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503006ace0 0x7f503006d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f504c14afe0 0x7f504c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50380043d0 0x7f504c15c020 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 --2- 172.21.15.27:0/3332297836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504c148a50 0x7f504c14aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 >> 172.21.15.27:0/3332297836 conn(0x7f504c06c150 msgr2=0x7f504c0b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.458+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 shutdown_connections 2023-07-06T20:14:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:46.459+0000 7f5053fff700 1 -- 172.21.15.27:0/3332297836 wait complete. 2023-07-06T20:14:46.478 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T20:14:46.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T20:14:46.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T20:14:46.683 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T20:14:46.683 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:46.684 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:46.730 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:46.731 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:46.816 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:46.817 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:46.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:47.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.148+0000 7fd515bff700 1 Processor -- start 2023-07-06T20:14:47.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.149+0000 7fd515bff700 1 -- start start 2023-07-06T20:14:47.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd515bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd51015fca0 0x7fd5100d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd515bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 0x7fd5100cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd515bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100cf780 con 0x7fd5100b5c90 2023-07-06T20:14:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd515bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100cd490 con 0x7fd51015fca0 2023-07-06T20:14:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd515bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100cd770 con 0x7fd5100b6110 2023-07-06T20:14:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd50effd700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd5100b5c90 0x7fd5100d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56722/0 (socket says 172.21.15.27:56722) 2023-07-06T20:14:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd50effd700 1 -- 172.21.15.27:0/523926870 learned_addr learned my addr 172.21.15.27:0/523926870 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/523926870 >> v2:172.21.15.153:3300/0 conn(0x7fd51015fca0 0x7fd5100d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.150+0000 7fd50ffff700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 0x7fd5100cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195331082 0 0) 0x7fd5100cf780 con 0x7fd5100b5c90 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f80035b0 con 0x7fd5100b5c90 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 825506349 0 0) 0x7fd4f80035b0 con 0x7fd5100b5c90 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 msgr2=0x7fd5100cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 0x7fd5100cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 >> v2:172.21.15.153:3300/0 conn(0x7fd51015fca0 msgr2=0x7fd5100d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 --2- 172.21.15.27:0/523926870 >> v2:172.21.15.153:3300/0 conn(0x7fd51015fca0 0x7fd5100d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100cda50 con 0x7fd5100b5c90 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7fd504001670 con 0x7fd5100b5c90 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 >> v1:172.21.15.27:6789/0 conn(0x7fd5100b5c90 legacy=0x7fd5100d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8004850 0x7fd4f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 0x7fd4f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100cd490 con 0x7fd4f8003a00 2023-07-06T20:14:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100cd770 con 0x7fd4f8004850 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.151+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800ba10 con 0x7fd5100b6110 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50effd700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8004850 0x7fd4f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50ffff700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 0x7fd4f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 -- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 msgr2=0x7fd4f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5100b6110 0x7fd4f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 -- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8004850 msgr2=0x7fd4f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8004850 0x7fd4f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 -- 172.21.15.27:0/523926870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80035b0 con 0x7fd4f8003a00 2023-07-06T20:14:47.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd500009b90 tx=0x7fd4f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c508076a1020bd3c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:47.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000e6c0 con 0x7fd4f8003a00 2023-07-06T20:14:47.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000e820 con 0x7fd4f8003a00 2023-07-06T20:14:47.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.152+0000 7fd50e7fc700 1 -- 172.21.15.27:0/523926870 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000eb90 con 0x7fd4f8003a00 2023-07-06T20:14:47.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd515bff700 1 -- 172.21.15.27:0/523926870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f8003a00 msgr2=0x7fd4f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd515bff700 1 --2- 172.21.15.27:0/523926870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4f8003a00 0x7fd4f8003e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd500009b90 tx=0x7fd4f8005880 comp rx=0 tx=0).stop 2023-07-06T20:14:47.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd50ffff700 1 -- 172.21.15.27:0/523926870 reap_dead start 2023-07-06T20:14:47.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd515bff700 1 -- 172.21.15.27:0/523926870 shutdown_connections 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd515bff700 1 -- 172.21.15.27:0/523926870 >> 172.21.15.27:0/523926870 conn(0x7fd51006c150 msgr2=0x7fd51006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd515bff700 1 -- 172.21.15.27:0/523926870 shutdown_connections 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.153+0000 7fd515bff700 1 -- 172.21.15.27:0/523926870 wait complete. 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.156+0000 7fd515bff700 1 Processor -- start 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.156+0000 7fd515bff700 1 -- start start 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd515bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8003a00 0x7fd51014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd515bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 0x7fd5100d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd515bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd51014c5b0 0x7fd5100d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd515bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51015bb30 con 0x7fd51014c140 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd515bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51015be10 con 0x7fd4f8003a00 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd515bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51015c0f0 con 0x7fd51014c5b0 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd50effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 0x7fd5100d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd50effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 0x7fd5100d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46522/0 (socket says 172.21.15.27:46522) 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.157+0000 7fd50effd700 1 -- 172.21.15.27:0/3462897675 learned_addr learned my addr 172.21.15.27:0/3462897675 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd50effd700 1 -- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd51014c5b0 msgr2=0x7fd5100d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd50effd700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd51014c5b0 0x7fd5100d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd50effd700 1 -- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8003a00 msgr2=0x7fd51014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd50effd700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8003a00 0x7fd51014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd50effd700 1 -- 172.21.15.27:0/3462897675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51015c3d0 con 0x7fd51014c140 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd50effd700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 0x7fd5100d0990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd504007fd0 tx=0x7fd5040075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400f640 con 0x7fd51014c140 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50400f7c0 con 0x7fd51014c140 2023-07-06T20:14:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400fb70 con 0x7fd51014c140 2023-07-06T20:14:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.158+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51015b7c0 con 0x7fd51014c140 2023-07-06T20:14:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.159+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd510159910 con 0x7fd51014c140 2023-07-06T20:14:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.160+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5100008d0 con 0x7fd51014c140 2023-07-06T20:14:47.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.161+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd50400fcd0 con 0x7fd51014c140 2023-07-06T20:14:47.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.162+0000 7fd4f7fff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd4fc06ad10 0x7fd4fc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.162+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd5040894c0 con 0x7fd51014c140 2023-07-06T20:14:47.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.162+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd4fc06ad10 0x7fd4fc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.163+0000 7fd50f7fe700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd4fc06ad10 0x7fd4fc06d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd500000c00 tx=0x7fd500013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:47.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.166+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd504054490 con 0x7fd51014c140 2023-07-06T20:14:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.378+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5100cde20 con 0x7fd51014c140 2023-07-06T20:14:47.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.379+0000 7fd4f7fff700 1 -- 172.21.15.27:0/3462897675 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7fd504057ab0 con 0x7fd51014c140 2023-07-06T20:14:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:47.382 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd4fc06ad10 msgr2=0x7fd4fc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd4fc06ad10 0x7fd4fc06d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd500000c00 tx=0x7fd500013000 comp rx=0 tx=0).stop 2023-07-06T20:14:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 msgr2=0x7fd5100d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 0x7fd5100d0990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd504007fd0 tx=0x7fd5040075f0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 shutdown_connections 2023-07-06T20:14:47.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd4fc06ad10 0x7fd4fc06d1c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd51014c5b0 0x7fd5100d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4f8003a00 0x7fd51014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 --2- 172.21.15.27:0/3462897675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd51014c140 0x7fd5100d0990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 >> 172.21.15.27:0/3462897675 conn(0x7fd51006c150 msgr2=0x7fd5100b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 shutdown_connections 2023-07-06T20:14:47.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.382+0000 7fd515bff700 1 -- 172.21.15.27:0/3462897675 wait complete. 2023-07-06T20:14:47.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:47.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:47.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.606+0000 7fce771ff700 1 Processor -- start 2023-07-06T20:14:47.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.606+0000 7fce771ff700 1 -- start start 2023-07-06T20:14:47.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.607+0000 7fce771ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce7015fca0 0x7fce700d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.607+0000 7fce771ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce700b5c90 0x7fce700d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.607+0000 7fce771ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700cf780 con 0x7fce700b6110 2023-07-06T20:14:47.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.607+0000 7fce771ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce700cd490 con 0x7fce7015fca0 2023-07-06T20:14:47.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.607+0000 7fce771ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce700cd770 con 0x7fce700b5c90 2023-07-06T20:14:47.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce74f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce7015fca0 0x7fce700d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce7579c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fce700b6110 0x7fce700cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56728/0 (socket says 172.21.15.27:56728) 2023-07-06T20:14:47.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce7579c700 1 -- 172.21.15.27:0/1505473303 learned_addr learned my addr 172.21.15.27:0/1505473303 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce6ffff700 1 --2- 172.21.15.27:0/1505473303 >> v2:172.21.15.153:3300/0 conn(0x7fce700b5c90 0x7fce700d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce74f9b700 1 -- 172.21.15.27:0/1505473303 >> v2:172.21.15.153:3300/0 conn(0x7fce700b5c90 msgr2=0x7fce700d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce74f9b700 1 --2- 172.21.15.27:0/1505473303 >> v2:172.21.15.153:3300/0 conn(0x7fce700b5c90 0x7fce700d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce74f9b700 1 -- 172.21.15.27:0/1505473303 >> v1:172.21.15.27:6789/0 conn(0x7fce700b6110 legacy=0x7fce700cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce74f9b700 1 -- 172.21.15.27:0/1505473303 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700cda50 con 0x7fce7015fca0 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce6f7fe700 1 -- 172.21.15.27:0/1505473303 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240814107 0 0) 0x7fce700cf780 con 0x7fce700b6110 2023-07-06T20:14:47.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.608+0000 7fce74f9b700 1 --2- 172.21.15.27:0/1505473303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7015fca0 0x7fce700d1420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce7006b6f0 tx=0x7fce64009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a3054a2ad82627 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce6f7fe700 1 -- 172.21.15.27:0/1505473303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64011830 con 0x7fce7015fca0 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce6f7fe700 1 -- 172.21.15.27:0/1505473303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64011990 con 0x7fce7015fca0 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce6f7fe700 1 -- 172.21.15.27:0/1505473303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64011d20 con 0x7fce7015fca0 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 -- 172.21.15.27:0/1505473303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7015fca0 msgr2=0x7fce700d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 --2- 172.21.15.27:0/1505473303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7015fca0 0x7fce700d1420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fce7006b6f0 tx=0x7fce64009b20 comp rx=0 tx=0).stop 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 -- 172.21.15.27:0/1505473303 shutdown_connections 2023-07-06T20:14:47.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 --2- 172.21.15.27:0/1505473303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7015fca0 0x7fce700d1420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 --2- 172.21.15.27:0/1505473303 >> v2:172.21.15.153:3300/0 conn(0x7fce700b5c90 0x7fce700d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 -- 172.21.15.27:0/1505473303 >> 172.21.15.27:0/1505473303 conn(0x7fce7006c150 msgr2=0x7fce7006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:47.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 -- 172.21.15.27:0/1505473303 shutdown_connections 2023-07-06T20:14:47.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.609+0000 7fce771ff700 1 -- 172.21.15.27:0/1505473303 wait complete. 2023-07-06T20:14:47.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.611+0000 7fce771ff700 1 Processor -- start 2023-07-06T20:14:47.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.611+0000 7fce771ff700 1 -- start start 2023-07-06T20:14:47.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce771ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 0x7fce700c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce771ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce700b6110 0x7fce700b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce771ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce7015fca0 0x7fce701493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce771ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70149910 con 0x7fce700b5c90 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce771ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700c6fc0 con 0x7fce700b6110 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce771ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700caa50 con 0x7fce7015fca0 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.612+0000 7fce74f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 0x7fce700c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 0x7fce700c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46542/0 (socket says 172.21.15.27:46542) 2023-07-06T20:14:47.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 -- 172.21.15.27:0/2811939143 learned_addr learned my addr 172.21.15.27:0/2811939143 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce7579c700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce7015fca0 0x7fce701493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce6ffff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce700b6110 0x7fce700b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 -- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce7015fca0 msgr2=0x7fce701493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce7015fca0 0x7fce701493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 -- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce700b6110 msgr2=0x7fce700b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce700b6110 0x7fce700b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 -- 172.21.15.27:0/2811939143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640094a0 con 0x7fce700b5c90 2023-07-06T20:14:47.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce74f9b700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 0x7fce700c6930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce64014040 tx=0x7fce64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:47.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.613+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640038a0 con 0x7fce700b5c90 2023-07-06T20:14:47.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.614+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6401e070 con 0x7fce700b5c90 2023-07-06T20:14:47.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.614+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce700cad30 con 0x7fce700b5c90 2023-07-06T20:14:47.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.614+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64003c20 con 0x7fce700b5c90 2023-07-06T20:14:47.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.614+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce700cb2f0 con 0x7fce700b5c90 2023-07-06T20:14:47.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.615+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce700008d0 con 0x7fce700b5c90 2023-07-06T20:14:47.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.616+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce6402d450 con 0x7fce700b5c90 2023-07-06T20:14:47.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.617+0000 7fce6d7fa700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fce5406ace0 0x7fce5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:47.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.617+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fce6408a460 con 0x7fce700b5c90 2023-07-06T20:14:47.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.617+0000 7fce6ffff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fce5406ace0 0x7fce5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:47.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.618+0000 7fce6ffff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fce5406ace0 0x7fce5406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce700d0d00 tx=0x7fce5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:47.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.621+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fce64055560 con 0x7fce700b5c90 2023-07-06T20:14:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.825+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce700d3a50 con 0x7fce700b5c90 2023-07-06T20:14:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.826+0000 7fce6d7fa700 1 -- 172.21.15.27:0/2811939143 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce64058b80 con 0x7fce700b5c90 2023-07-06T20:14:47.827 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fce5406ace0 msgr2=0x7fce5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fce5406ace0 0x7fce5406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce700d0d00 tx=0x7fce5c008040 comp rx=0 tx=0).stop 2023-07-06T20:14:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 msgr2=0x7fce700c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 0x7fce700c6930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce64014040 tx=0x7fce64004750 comp rx=0 tx=0).stop 2023-07-06T20:14:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 shutdown_connections 2023-07-06T20:14:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fce5406ace0 0x7fce5406d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fce7015fca0 0x7fce701493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce700b5c90 0x7fce700c6930 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 --2- 172.21.15.27:0/2811939143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce700b6110 0x7fce700b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:47.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 >> 172.21.15.27:0/2811939143 conn(0x7fce7006c150 msgr2=0x7fce700d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:47.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 shutdown_connections 2023-07-06T20:14:47.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:47.828+0000 7fce771ff700 1 -- 172.21.15.27:0/2811939143 wait complete. 2023-07-06T20:14:47.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.056+0000 7f62729ff700 1 Processor -- start 2023-07-06T20:14:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.057+0000 7f62729ff700 1 -- start start 2023-07-06T20:14:48.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f62729ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f626c15fca0 0x7f626c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f62729ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c0cf780 con 0x7f626c0b6110 2023-07-06T20:14:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f62729ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c0cd490 con 0x7f626c0b5c90 2023-07-06T20:14:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f62729ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626c0cd770 con 0x7f626c15fca0 2023-07-06T20:14:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f6270f9c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f626c0b6110 0x7f626c0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56738/0 (socket says 172.21.15.27:56738) 2023-07-06T20:14:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f6270f9c700 1 -- 172.21.15.27:0/4106916508 learned_addr learned my addr 172.21.15.27:0/4106916508 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.058+0000 7f626bfff700 1 --2- 172.21.15.27:0/4106916508 >> v2:172.21.15.153:3300/0 conn(0x7f626c15fca0 0x7f626c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.059+0000 7f626affd700 1 -- 172.21.15.27:0/4106916508 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534645677 0 0) 0x7f626c0cf780 con 0x7f626c0b6110 2023-07-06T20:14:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.059+0000 7f626affd700 1 -- 172.21.15.27:0/4106916508 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62540035b0 con 0x7f626c0b6110 2023-07-06T20:14:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.059+0000 7f626bfff700 1 -- 172.21.15.27:0/4106916508 >> v1:172.21.15.153:6789/0 conn(0x7f626c0b5c90 legacy=0x7f626c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.059+0000 7f626bfff700 1 -- 172.21.15.27:0/4106916508 >> v1:172.21.15.27:6789/0 conn(0x7f626c0b6110 legacy=0x7f626c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.059+0000 7f626bfff700 1 -- 172.21.15.27:0/4106916508 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0cda50 con 0x7f626c15fca0 2023-07-06T20:14:48.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.059+0000 7f626bfff700 1 --2- 172.21.15.27:0/4106916508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c0d1420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f625c009840 tx=0x7f625c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f3f6b11bc54c10 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f626affd700 1 -- 172.21.15.27:0/4106916508 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c011830 con 0x7f626c15fca0 2023-07-06T20:14:48.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f626affd700 1 -- 172.21.15.27:0/4106916508 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c011990 con 0x7f626c15fca0 2023-07-06T20:14:48.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f626affd700 1 -- 172.21.15.27:0/4106916508 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c011d20 con 0x7f626c15fca0 2023-07-06T20:14:48.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 -- 172.21.15.27:0/4106916508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 msgr2=0x7f626c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 --2- 172.21.15.27:0/4106916508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c0d1420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f625c009840 tx=0x7f625c009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:48.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 -- 172.21.15.27:0/4106916508 shutdown_connections 2023-07-06T20:14:48.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 --2- 172.21.15.27:0/4106916508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c0d1420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 -- 172.21.15.27:0/4106916508 >> 172.21.15.27:0/4106916508 conn(0x7f626c06c150 msgr2=0x7f626c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:48.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 -- 172.21.15.27:0/4106916508 shutdown_connections 2023-07-06T20:14:48.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.060+0000 7f62729ff700 1 -- 172.21.15.27:0/4106916508 wait complete. 2023-07-06T20:14:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.062+0000 7f62729ff700 1 Processor -- start 2023-07-06T20:14:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.062+0000 7f62729ff700 1 -- start start 2023-07-06T20:14:48.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f62729ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 0x7f626c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f62729ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f626c0b6110 0x7f626c0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f62729ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f62729ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1570e0 con 0x7f626c0b5c90 2023-07-06T20:14:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f62729ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1564d0 con 0x7f626c15fca0 2023-07-06T20:14:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f62729ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1567b0 con 0x7f626c0b6110 2023-07-06T20:14:48.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f626bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 0x7f626c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f626bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 0x7f626c1513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46546/0 (socket says 172.21.15.27:46546) 2023-07-06T20:14:48.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f626bfff700 1 -- 172.21.15.27:0/2123055584 learned_addr learned my addr 172.21.15.27:0/2123055584 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f626b7fe700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f626c0b6110 0x7f626c0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.064+0000 7f6270f9c700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f626bfff700 1 -- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f626c0b6110 msgr2=0x7f626c0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f626bfff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f626c0b6110 0x7f626c0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f626bfff700 1 -- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 msgr2=0x7f626c158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f626bfff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f626bfff700 1 -- 172.21.15.27:0/2123055584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0094a0 con 0x7f626c0b5c90 2023-07-06T20:14:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f626bfff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 0x7f626c1513a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f625c009b90 tx=0x7f625c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0036a0 con 0x7f626c0b5c90 2023-07-06T20:14:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c01e070 con 0x7f626c0b5c90 2023-07-06T20:14:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c156a90 con 0x7f626c0b5c90 2023-07-06T20:14:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.065+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c003a60 con 0x7f626c0b5c90 2023-07-06T20:14:48.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.066+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c1518e0 con 0x7f626c0b5c90 2023-07-06T20:14:48.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.067+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c0008d0 con 0x7f626c0b5c90 2023-07-06T20:14:48.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.068+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f625c003bc0 con 0x7f626c0b5c90 2023-07-06T20:14:48.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.069+0000 7f6268ff9700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f625806ace0 0x7f625806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.069+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f625c08a370 con 0x7f626c0b5c90 2023-07-06T20:14:48.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.069+0000 7f626b7fe700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f625806ace0 0x7f625806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.070+0000 7f626b7fe700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f625806ace0 0x7f625806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f626c0c6f40 tx=0x7f6260009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.073+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f625c0553d0 con 0x7f626c0b5c90 2023-07-06T20:14:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.280+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f626c0cd150 con 0x7f626c0b5c90 2023-07-06T20:14:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.280+0000 7f6268ff9700 1 -- 172.21.15.27:0/2123055584 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f625c0589f0 con 0x7f626c0b5c90 2023-07-06T20:14:48.282 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.282+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f625806ace0 msgr2=0x7f625806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.282+0000 7f62729ff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f625806ace0 0x7f625806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f626c0c6f40 tx=0x7f6260009040 comp rx=0 tx=0).stop 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.282+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 msgr2=0x7f626c1513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.282+0000 7f62729ff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 0x7f626c1513a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f625c009b90 tx=0x7f625c004750 comp rx=0 tx=0).stop 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.282+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 shutdown_connections 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f625806ace0 0x7f625806d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f626c15fca0 0x7f626c158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f626c0b5c90 0x7f626c1513a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 --2- 172.21.15.27:0/2123055584 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f626c0b6110 0x7f626c0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 >> 172.21.15.27:0/2123055584 conn(0x7f626c06c150 msgr2=0x7f626c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 shutdown_connections 2023-07-06T20:14:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.283+0000 7f62729ff700 1 -- 172.21.15.27:0/2123055584 wait complete. 2023-07-06T20:14:48.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:48.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.507+0000 7f8ba35ff700 1 Processor -- start 2023-07-06T20:14:48.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.507+0000 7f8ba35ff700 1 -- start start 2023-07-06T20:14:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.508+0000 7f8ba35ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b9c0b5c90 0x7f8b9c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.508+0000 7f8ba35ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b6110 0x7f8b9c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.508+0000 7f8ba35ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9c0cf780 con 0x7f8b9c15fca0 2023-07-06T20:14:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.508+0000 7f8ba35ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9c0cd490 con 0x7f8b9c0b5c90 2023-07-06T20:14:48.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.508+0000 7f8ba35ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0cd770 con 0x7f8b9c0b6110 2023-07-06T20:14:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b9c0b5c90 0x7f8b9c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b9c0b5c90 0x7f8b9c0d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46554/0 (socket says 172.21.15.27:46554) 2023-07-06T20:14:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/3459542894 learned_addr learned my addr 172.21.15.27:0/3459542894 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba1b9c700 1 --2- 172.21.15.27:0/3459542894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b6110 0x7f8b9c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/3459542894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b6110 msgr2=0x7f8b9c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 --2- 172.21.15.27:0/3459542894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b6110 0x7f8b9c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/3459542894 >> v1:172.21.15.153:6789/0 conn(0x7f8b9c15fca0 legacy=0x7f8b9c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/3459542894 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c0cda50 con 0x7f8b9c0b5c90 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.509+0000 7f8ba0b9a700 1 --2- 172.21.15.27:0/3459542894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b5c90 0x7f8b9c0d2d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098f0 tx=0x7f8b8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91d4634dd6624fb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8b93fff700 1 -- 172.21.15.27:0/3459542894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00b8a0 con 0x7f8b9c0b5c90 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8b93fff700 1 -- 172.21.15.27:0/3459542894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c00ba00 con 0x7f8b9c0b5c90 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8b93fff700 1 -- 172.21.15.27:0/3459542894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00bd90 con 0x7f8b9c0b5c90 2023-07-06T20:14:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 -- 172.21.15.27:0/3459542894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b5c90 msgr2=0x7f8b9c0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/3459542894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b5c90 0x7f8b9c0d2d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098f0 tx=0x7f8b8c009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 -- 172.21.15.27:0/3459542894 shutdown_connections 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/3459542894 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b6110 0x7f8b9c0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/3459542894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b5c90 0x7f8b9c0d2d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 -- 172.21.15.27:0/3459542894 >> 172.21.15.27:0/3459542894 conn(0x7f8b9c06c150 msgr2=0x7f8b9c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 -- 172.21.15.27:0/3459542894 shutdown_connections 2023-07-06T20:14:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.510+0000 7f8ba35ff700 1 -- 172.21.15.27:0/3459542894 wait complete. 2023-07-06T20:14:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.512+0000 7f8ba35ff700 1 Processor -- start 2023-07-06T20:14:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.512+0000 7f8ba35ff700 1 -- start start 2023-07-06T20:14:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba35ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b5c90 0x7f8b9c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba35ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 0x7f8b9c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba35ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b9c15fca0 0x7f8b9c0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba35ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0c9f10 con 0x7f8b9c0b6110 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba35ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0cbed0 con 0x7f8b9c15fca0 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba35ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0ca160 con 0x7f8b9c0b5c90 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba0b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 0x7f8b9c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba0b9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 0x7f8b9c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46566/0 (socket says 172.21.15.27:46566) 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.513+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/4249274282 learned_addr learned my addr 172.21.15.27:0/4249274282 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba139b700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b5c90 0x7f8b9c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba1b9c700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b9c15fca0 0x7f8b9c0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b5c90 msgr2=0x7f8b9c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba0b9a700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b5c90 0x7f8b9c0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b9c15fca0 msgr2=0x7f8b9c0b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba0b9a700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b9c15fca0 0x7f8b9c0b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba0b9a700 1 -- 172.21.15.27:0/4249274282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009550 con 0x7f8b9c0b6110 2023-07-06T20:14:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba0b9a700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 0x7f8b9c156840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c004480 con 0x7f8b9c0b6110 2023-07-06T20:14:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c01f070 con 0x7f8b9c0b6110 2023-07-06T20:14:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c0ca440 con 0x7f8b9c0b6110 2023-07-06T20:14:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.514+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0038c0 con 0x7f8b9c0b6110 2023-07-06T20:14:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.515+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c0cbac0 con 0x7f8b9c0b6110 2023-07-06T20:14:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.515+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9c0ca960 con 0x7f8b9c0b6110 2023-07-06T20:14:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.516+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b8c021030 con 0x7f8b9c0b6110 2023-07-06T20:14:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.517+0000 7f8b91ffb700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8b8006ace0 0x7f8b8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.517+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8b8c08aaf0 con 0x7f8b9c0b6110 2023-07-06T20:14:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.518+0000 7f8ba139b700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8b8006ace0 0x7f8b8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.519+0000 7f8ba139b700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8b8006ace0 0x7f8b8006d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b98000f60 tx=0x7f8b98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.522+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b9c0ca960 con 0x7f8b9c0b6110 2023-07-06T20:14:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.727+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b9c0ca960 con 0x7f8b9c0b6110 2023-07-06T20:14:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.727+0000 7f8b91ffb700 1 -- 172.21.15.27:0/4249274282 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b9c0ca960 con 0x7f8b9c0b6110 2023-07-06T20:14:48.729 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8b8006ace0 msgr2=0x7f8b8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8b8006ace0 0x7f8b8006d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b98000f60 tx=0x7f8b98009040 comp rx=0 tx=0).stop 2023-07-06T20:14:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 msgr2=0x7f8b9c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 0x7f8b9c156840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c004750 comp rx=0 tx=0).stop 2023-07-06T20:14:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 shutdown_connections 2023-07-06T20:14:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8b8006ace0 0x7f8b8006d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b9c15fca0 0x7f8b9c0b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8b9c0b5c90 0x7f8b9c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 --2- 172.21.15.27:0/4249274282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0b6110 0x7f8b9c156840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 >> 172.21.15.27:0/4249274282 conn(0x7f8b9c06c150 msgr2=0x7f8b9c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 shutdown_connections 2023-07-06T20:14:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.730+0000 7f8ba35ff700 1 -- 172.21.15.27:0/4249274282 wait complete. 2023-07-06T20:14:48.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:48.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.954+0000 7f1cc0fff700 1 Processor -- start 2023-07-06T20:14:48.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.955+0000 7f1cc0fff700 1 -- start start 2023-07-06T20:14:48.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cc0fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc15fca0 0x7f1cbc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cc0fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0cf780 con 0x7f1cbc0b6100 2023-07-06T20:14:48.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cc0fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0cd490 con 0x7f1cbc0b5c90 2023-07-06T20:14:48.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cc0fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0cd770 con 0x7f1cbc15fca0 2023-07-06T20:14:48.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cbad9d700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1cbc0b6100 0x7f1cbc0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56754/0 (socket says 172.21.15.27:56754) 2023-07-06T20:14:48.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cbad9d700 1 -- 172.21.15.27:0/1261549560 learned_addr learned my addr 172.21.15.27:0/1261549560 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:48.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.956+0000 7f1cba59c700 1 --2- 172.21.15.27:0/1261549560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc15fca0 0x7f1cbc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cb9d9b700 1 -- 172.21.15.27:0/1261549560 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254541514 0 0) 0x7f1cbc0cf780 con 0x7f1cbc0b6100 2023-07-06T20:14:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cb9d9b700 1 -- 172.21.15.27:0/1261549560 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca80035b0 con 0x7f1cbc0b6100 2023-07-06T20:14:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cba59c700 1 -- 172.21.15.27:0/1261549560 >> v1:172.21.15.153:6789/0 conn(0x7f1cbc0b5c90 legacy=0x7f1cbc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cb9d9b700 1 -- 172.21.15.27:0/1261549560 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192766510 0 0) 0x7f1cbc0cd490 con 0x7f1cbc0b5c90 2023-07-06T20:14:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cba59c700 1 -- 172.21.15.27:0/1261549560 >> v1:172.21.15.27:6789/0 conn(0x7f1cbc0b6100 legacy=0x7f1cbc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cba59c700 1 -- 172.21.15.27:0/1261549560 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc0cda50 con 0x7f1cbc15fca0 2023-07-06T20:14:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cba59c700 1 --2- 172.21.15.27:0/1261549560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc15fca0 0x7f1cbc0d1420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009870 tx=0x7f1cac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f666edd5e57b5fa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.957+0000 7f1cb9d9b700 1 -- 172.21.15.27:0/1261549560 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011830 con 0x7f1cbc15fca0 2023-07-06T20:14:48.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cb9d9b700 1 -- 172.21.15.27:0/1261549560 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac011990 con 0x7f1cbc15fca0 2023-07-06T20:14:48.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cb9d9b700 1 -- 172.21.15.27:0/1261549560 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011d20 con 0x7f1cbc15fca0 2023-07-06T20:14:48.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 -- 172.21.15.27:0/1261549560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc15fca0 msgr2=0x7f1cbc0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/1261549560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc15fca0 0x7f1cbc0d1420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009870 tx=0x7f1cac009b80 comp rx=0 tx=0).stop 2023-07-06T20:14:48.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 -- 172.21.15.27:0/1261549560 shutdown_connections 2023-07-06T20:14:48.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/1261549560 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc15fca0 0x7f1cbc0d1420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 -- 172.21.15.27:0/1261549560 >> 172.21.15.27:0/1261549560 conn(0x7f1cbc06c150 msgr2=0x7f1cbc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:48.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 -- 172.21.15.27:0/1261549560 shutdown_connections 2023-07-06T20:14:48.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.958+0000 7f1cc0fff700 1 -- 172.21.15.27:0/1261549560 wait complete. 2023-07-06T20:14:48.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.960+0000 7f1cc0fff700 1 Processor -- start 2023-07-06T20:14:48.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.960+0000 7f1cc0fff700 1 -- start start 2023-07-06T20:14:48.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cc0fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc0b5c90 0x7f1cbc1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cc0fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 0x7f1cbc0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cc0fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc15fca0 0x7f1cbc0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cc0fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c9f10 con 0x7f1cbc0b6100 2023-07-06T20:14:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cc0fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc1f2bb0 con 0x7f1cbc15fca0 2023-07-06T20:14:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cc0fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0ca1b0 con 0x7f1cbc0b5c90 2023-07-06T20:14:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cb3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 0x7f1cbc0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cb3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 0x7f1cbc0c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46580/0 (socket says 172.21.15.27:46580) 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cb3fff700 1 -- 172.21.15.27:0/592493819 learned_addr learned my addr 172.21.15.27:0/592493819 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.961+0000 7f1cba59c700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc0b5c90 0x7f1cbc1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cbad9d700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc15fca0 0x7f1cbc0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb3fff700 1 -- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc0b5c90 msgr2=0x7f1cbc1f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb3fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc0b5c90 0x7f1cbc1f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb3fff700 1 -- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc15fca0 msgr2=0x7f1cbc0b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb3fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc15fca0 0x7f1cbc0b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb3fff700 1 -- 172.21.15.27:0/592493819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0094a0 con 0x7f1cbc0b6100 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cbad9d700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc15fca0 0x7f1cbc0b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb3fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 0x7f1cbc0c8550 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40011e0 tx=0x7f1ca400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40054e0 con 0x7f1cbc0b6100 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca4003680 con 0x7f1cbc0b6100 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4003a30 con 0x7f1cbc0b6100 2023-07-06T20:14:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc0ca490 con 0x7f1cbc0b6100 2023-07-06T20:14:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.962+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc0caa50 con 0x7f1cbc0b6100 2023-07-06T20:14:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.963+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc0008d0 con 0x7f1cbc0b6100 2023-07-06T20:14:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.964+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ca4003b90 con 0x7f1cbc0b6100 2023-07-06T20:14:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.965+0000 7f1cb2ffd700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1c9c06ace0 0x7f1c9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.965+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1ca4088ad0 con 0x7f1cbc0b6100 2023-07-06T20:14:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.966+0000 7f1cba59c700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1c9c06ace0 0x7f1c9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.967+0000 7f1cba59c700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1c9c06ace0 0x7f1c9c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0097c0 tx=0x7f1cac01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:48.970+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1ca4053aa0 con 0x7f1cbc0b6100 2023-07-06T20:14:49.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.179+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cbc06aa40 con 0x7f1cbc0b6100 2023-07-06T20:14:49.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.179+0000 7f1cb2ffd700 1 -- 172.21.15.27:0/592493819 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ca4017020 con 0x7f1cbc0b6100 2023-07-06T20:14:49.181 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:49.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1c9c06ace0 msgr2=0x7f1c9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1c9c06ace0 0x7f1c9c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0097c0 tx=0x7f1cac01c040 comp rx=0 tx=0).stop 2023-07-06T20:14:49.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 msgr2=0x7f1cbc0c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 0x7f1cbc0c8550 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40011e0 tx=0x7f1ca400cae0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 shutdown_connections 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1c9c06ace0 0x7f1c9c06d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc15fca0 0x7f1cbc0b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cbc0b5c90 0x7f1cbc1f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 --2- 172.21.15.27:0/592493819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0b6100 0x7f1cbc0c8550 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 >> 172.21.15.27:0/592493819 conn(0x7f1cbc06c150 msgr2=0x7f1cbc0d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 shutdown_connections 2023-07-06T20:14:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.182+0000 7f1cc0fff700 1 -- 172.21.15.27:0/592493819 wait complete. 2023-07-06T20:14:49.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:14:49.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.406+0000 7f4e493ff700 1 Processor -- start 2023-07-06T20:14:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.406+0000 7f4e493ff700 1 -- start start 2023-07-06T20:14:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e493ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e440b0440 0x7f4e44143900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e493ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4e4415df60 0x7f4e440b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e493ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e4415b0e0 0x7f4e4415b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e493ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e441441f0 con 0x7f4e4415df60 2023-07-06T20:14:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e493ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4415a930 con 0x7f4e4415b0e0 2023-07-06T20:14:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e493ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4415e3d0 con 0x7f4e440b0440 2023-07-06T20:14:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e427fc700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4e4415df60 0x7f4e440b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e427fc700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4e4415df60 0x7f4e440b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46582/0 (socket says 172.21.15.27:46582) 2023-07-06T20:14:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e427fc700 1 -- 172.21.15.27:0/3284017041 learned_addr learned my addr 172.21.15.27:0/3284017041 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e42ffd700 1 --2- 172.21.15.27:0/3284017041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e440b0440 0x7f4e44143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.407+0000 7f4e437fe700 1 --2- 172.21.15.27:0/3284017041 >> v2:172.21.15.153:3300/0 conn(0x7f4e4415b0e0 0x7f4e4415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e427fc700 1 -- 172.21.15.27:0/3284017041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e440b0440 msgr2=0x7f4e44143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e427fc700 1 --2- 172.21.15.27:0/3284017041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e440b0440 0x7f4e44143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e427fc700 1 -- 172.21.15.27:0/3284017041 >> v2:172.21.15.153:3300/0 conn(0x7f4e4415b0e0 msgr2=0x7f4e4415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e427fc700 1 --2- 172.21.15.27:0/3284017041 >> v2:172.21.15.153:3300/0 conn(0x7f4e4415b0e0 0x7f4e4415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e427fc700 1 -- 172.21.15.27:0/3284017041 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440b0d50 con 0x7f4e4415df60 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e427fc700 1 --2- 172.21.15.27:0/3284017041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e4415df60 0x7f4e440b0810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e44071c00 tx=0x7f4e38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa7329a833b7637 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e41ffb700 1 -- 172.21.15.27:0/3284017041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011830 con 0x7f4e4415df60 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e41ffb700 1 -- 172.21.15.27:0/3284017041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e38011990 con 0x7f4e4415df60 2023-07-06T20:14:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.408+0000 7f4e41ffb700 1 -- 172.21.15.27:0/3284017041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011d20 con 0x7f4e4415df60 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 -- 172.21.15.27:0/3284017041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e4415df60 msgr2=0x7f4e440b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 --2- 172.21.15.27:0/3284017041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e4415df60 0x7f4e440b0810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e44071c00 tx=0x7f4e38009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 -- 172.21.15.27:0/3284017041 shutdown_connections 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 --2- 172.21.15.27:0/3284017041 >> v2:172.21.15.153:3300/0 conn(0x7f4e4415b0e0 0x7f4e4415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 --2- 172.21.15.27:0/3284017041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e440b0440 0x7f4e44143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 --2- 172.21.15.27:0/3284017041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e4415df60 0x7f4e440b0810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 -- 172.21.15.27:0/3284017041 >> 172.21.15.27:0/3284017041 conn(0x7f4e440cb440 msgr2=0x7f4e440cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 -- 172.21.15.27:0/3284017041 shutdown_connections 2023-07-06T20:14:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.409+0000 7f4e493ff700 1 -- 172.21.15.27:0/3284017041 wait complete. 2023-07-06T20:14:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.411+0000 7f4e493ff700 1 Processor -- start 2023-07-06T20:14:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.411+0000 7f4e493ff700 1 -- start start 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e493ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 0x7f4e440b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e493ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e4415b0e0 0x7f4e44157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e493ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4415df60 0x7f4e4406c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e493ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e440b2550 con 0x7f4e440b0440 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e493ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441e4c90 con 0x7f4e4415df60 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e493ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e44157b80 con 0x7f4e4415b0e0 2023-07-06T20:14:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.412+0000 7f4e42ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 0x7f4e440b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 0x7f4e440b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46594/0 (socket says 172.21.15.27:46594) 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 -- 172.21.15.27:0/101202534 learned_addr learned my addr 172.21.15.27:0/101202534 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e427fc700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e4415b0e0 0x7f4e44157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e437fe700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4415df60 0x7f4e4406c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 -- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e4415b0e0 msgr2=0x7f4e44157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e4415b0e0 0x7f4e44157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 -- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4415df60 msgr2=0x7f4e4406c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4415df60 0x7f4e4406c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 -- 172.21.15.27:0/101202534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38009550 con 0x7f4e440b0440 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.413+0000 7f4e42ffd700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 0x7f4e440b3540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400e870 tx=0x7f4e3400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.414+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34013680 con 0x7f4e440b0440 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.414+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e340137e0 con 0x7f4e440b0440 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.414+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34013b90 con 0x7f4e440b0440 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.414+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e440b2780 con 0x7f4e440b0440 2023-07-06T20:14:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.414+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e440b2d40 con 0x7f4e440b0440 2023-07-06T20:14:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.415+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4406fa50 con 0x7f4e440b0440 2023-07-06T20:14:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.416+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e34013cf0 con 0x7f4e440b0440 2023-07-06T20:14:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.417+0000 7f4e2b7fe700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4e3006ace0 0x7f4e3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.417+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4e34089310 con 0x7f4e440b0440 2023-07-06T20:14:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.417+0000 7f4e427fc700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4e3006ace0 0x7f4e3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.418+0000 7f4e427fc700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4e3006ace0 0x7f4e3006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009c10 tx=0x7f4e38008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.421+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4e340542e0 con 0x7f4e440b0440 2023-07-06T20:14:49.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.642+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e440008d0 con 0x7f4e440b0440 2023-07-06T20:14:49.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.643+0000 7f4e2b7fe700 1 -- 172.21.15.27:0/101202534 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e34057900 con 0x7f4e440b0440 2023-07-06T20:14:49.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4e3006ace0 msgr2=0x7f4e3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4e3006ace0 0x7f4e3006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009c10 tx=0x7f4e38008030 comp rx=0 tx=0).stop 2023-07-06T20:14:49.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 msgr2=0x7f4e440b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:49.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 0x7f4e440b3540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400e870 tx=0x7f4e3400eb80 comp rx=0 tx=0).stop 2023-07-06T20:14:49.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 shutdown_connections 2023-07-06T20:14:49.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4e3006ace0 0x7f4e3006d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4415df60 0x7f4e4406c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e440b0440 0x7f4e440b3540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 --2- 172.21.15.27:0/101202534 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4e4415b0e0 0x7f4e44157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 >> 172.21.15.27:0/101202534 conn(0x7f4e440cb440 msgr2=0x7f4e440ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 shutdown_connections 2023-07-06T20:14:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:49.645+0000 7f4e493ff700 1 -- 172.21.15.27:0/101202534 wait complete. 2023-07-06T20:14:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-07-06T20:14:49.665 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T20:14:49.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:49.666 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-07-06T20:14:49.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:14:49.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:49.752 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T20:14:49.813 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:49.813 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:49.863 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:49.863 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:49.910 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:49.911 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:49.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:50.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.200+0000 7f504efff700 1 Processor -- start 2023-07-06T20:14:50.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.200+0000 7f504efff700 1 -- start start 2023-07-06T20:14:50.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.201+0000 7f504efff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f504815fca0 0x7f50480d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504efff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 0x7f50480cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504efff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50480cf780 con 0x7f50480b5c90 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504efff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50480cd490 con 0x7f504815fca0 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504efff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50480cd770 con 0x7f50480b6110 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504d59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 0x7f50480cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504d59c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 0x7f50480cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:57636/0 (socket says 172.21.15.27:57636) 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504d59c700 1 -- 172.21.15.27:0/3674909234 learned_addr learned my addr 172.21.15.27:0/3674909234 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:50.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504cd9b700 1 --2- 172.21.15.27:0/3674909234 >> v2:172.21.15.153:3300/0 conn(0x7f504815fca0 0x7f50480d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.202+0000 7f504d59c700 1 -- 172.21.15.27:0/3674909234 >> v2:172.21.15.153:3300/0 conn(0x7f504815fca0 msgr2=0x7f50480d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:50.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f504d59c700 1 --2- 172.21.15.27:0/3674909234 >> v2:172.21.15.153:3300/0 conn(0x7f504815fca0 0x7f50480d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f504d59c700 1 -- 172.21.15.27:0/3674909234 >> v1:172.21.15.153:6789/0 conn(0x7f50480b5c90 legacy=0x7f50480d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f504d59c700 1 -- 172.21.15.27:0/3674909234 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50480cda50 con 0x7f50480b6110 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f504d59c700 1 --2- 172.21.15.27:0/3674909234 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 0x7f50480cfc10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5040009e50 tx=0x7f504000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed015e5835c7ec0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f50477fe700 1 -- 172.21.15.27:0/3674909234 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000e070 con 0x7f50480b6110 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f50477fe700 1 -- 172.21.15.27:0/3674909234 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504000b910 con 0x7f50480b6110 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.203+0000 7f50477fe700 1 -- 172.21.15.27:0/3674909234 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000bcc0 con 0x7f50480b6110 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 -- 172.21.15.27:0/3674909234 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 msgr2=0x7f50480cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 --2- 172.21.15.27:0/3674909234 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 0x7f50480cfc10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5040009e50 tx=0x7f504000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 -- 172.21.15.27:0/3674909234 shutdown_connections 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 --2- 172.21.15.27:0/3674909234 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50480b6110 0x7f50480cfc10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 --2- 172.21.15.27:0/3674909234 >> v2:172.21.15.153:3300/0 conn(0x7f504815fca0 0x7f50480d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 -- 172.21.15.27:0/3674909234 >> 172.21.15.27:0/3674909234 conn(0x7f504806c150 msgr2=0x7f504806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 -- 172.21.15.27:0/3674909234 shutdown_connections 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.204+0000 7f504efff700 1 -- 172.21.15.27:0/3674909234 wait complete. 2023-07-06T20:14:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.206+0000 7f504efff700 1 Processor -- start 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.206+0000 7f504efff700 1 -- start start 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504efff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 0x7f5048156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504efff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50480b6110 0x7f50480c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504efff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504815fca0 0x7f504815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504efff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50480c7eb0 con 0x7f50480b6110 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504efff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50480c72a0 con 0x7f50480b5c90 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504efff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50480c7580 con 0x7f504815fca0 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f5047fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50480b6110 0x7f50480c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504cd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 0x7f5048156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504cd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 0x7f5048156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:46948/0 (socket says 172.21.15.27:46948) 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504cd9b700 1 -- 172.21.15.27:0/1132275343 learned_addr learned my addr 172.21.15.27:0/1132275343 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.207+0000 7f504d59c700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504815fca0 0x7f504815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504cd9b700 1 -- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504815fca0 msgr2=0x7f504815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504cd9b700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504815fca0 0x7f504815c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504cd9b700 1 -- 172.21.15.27:0/1132275343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50480b6110 msgr2=0x7f50480c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504cd9b700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50480b6110 0x7f50480c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504cd9b700 1 -- 172.21.15.27:0/1132275343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009b30 con 0x7f50480b5c90 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f5047fff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50480b6110 0x7f50480c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504cd9b700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 0x7f5048156b80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f503c00e870 tx=0x7f503c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c009590 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c013680 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50480c7860 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c0139f0 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.208+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50481570c0 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.209+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50480008d0 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.211+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f503c013b50 con 0x7f50480b5c90 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.212+0000 7f50457fa700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f502c06ace0 0x7f502c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.212+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f503c0893a0 con 0x7f50480b5c90 2023-07-06T20:14:50.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.212+0000 7f5047fff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f502c06ace0 0x7f502c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.213+0000 7f5047fff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f502c06ace0 0x7f502c06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f50481f57f0 tx=0x7f5034009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.215+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f503c057f10 con 0x7f50480b5c90 2023-07-06T20:14:50.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.422+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50480b6c20 con 0x7f50480b5c90 2023-07-06T20:14:50.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.423+0000 7f50457fa700 1 -- 172.21.15.27:0/1132275343 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f503c057aa0 con 0x7f50480b5c90 2023-07-06T20:14:50.425 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:50.425 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f502c06ace0 msgr2=0x7f502c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f502c06ace0 0x7f502c06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f50481f57f0 tx=0x7f5034009040 comp rx=0 tx=0).stop 2023-07-06T20:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 msgr2=0x7f5048156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 0x7f5048156b80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f503c00e870 tx=0x7f503c00eb80 comp rx=0 tx=0).stop 2023-07-06T20:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 shutdown_connections 2023-07-06T20:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f502c06ace0 0x7f502c06d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f504815fca0 0x7f504815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50480b5c90 0x7f5048156b80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 --2- 172.21.15.27:0/1132275343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50480b6110 0x7f50480c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 >> 172.21.15.27:0/1132275343 conn(0x7f504806c150 msgr2=0x7f50480cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 shutdown_connections 2023-07-06T20:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.426+0000 7f504efff700 1 -- 172.21.15.27:0/1132275343 wait complete. 2023-07-06T20:14:50.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:50.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:50.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:14:50.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:14:50.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.653+0000 7f049b9ff700 1 Processor -- start 2023-07-06T20:14:50.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.653+0000 7f049b9ff700 1 -- start start 2023-07-06T20:14:50.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04940b5c90 0x7f04940d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049b9ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f04940b60e0 0x7f04940d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049b9ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04940cf780 con 0x7f049415fca0 2023-07-06T20:14:50.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049b9ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940cd490 con 0x7f04940b60e0 2023-07-06T20:14:50.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940cd770 con 0x7f04940b5c90 2023-07-06T20:14:50.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049979b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f049415fca0 0x7f04940d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56762/0 (socket says 172.21.15.27:56762) 2023-07-06T20:14:50.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f049979b700 1 -- 172.21.15.27:0/2072266074 learned_addr learned my addr 172.21.15.27:0/2072266074 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:50.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f0498f9a700 1 --2- 172.21.15.27:0/2072266074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04940b5c90 0x7f04940d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.654+0000 7f0499f9c700 1 --2- 172.21.15.27:0/2072266074 >> v2:172.21.15.153:3300/0 conn(0x7f04940b60e0 0x7f04940d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f048bfff700 1 -- 172.21.15.27:0/2072266074 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579944858 0 0) 0x7f04940cf780 con 0x7f049415fca0 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f048bfff700 1 -- 172.21.15.27:0/2072266074 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04800035b0 con 0x7f049415fca0 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f0498f9a700 1 -- 172.21.15.27:0/2072266074 >> v2:172.21.15.153:3300/0 conn(0x7f04940b60e0 msgr2=0x7f04940d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f0498f9a700 1 --2- 172.21.15.27:0/2072266074 >> v2:172.21.15.153:3300/0 conn(0x7f04940b60e0 0x7f04940d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f0498f9a700 1 -- 172.21.15.27:0/2072266074 >> v1:172.21.15.27:6789/0 conn(0x7f049415fca0 legacy=0x7f04940d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f0498f9a700 1 -- 172.21.15.27:0/2072266074 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940cda50 con 0x7f04940b5c90 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f0498f9a700 1 --2- 172.21.15.27:0/2072266074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04940b5c90 0x7f04940d2d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0484009840 tx=0x7f0484009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a32ba9680251e4a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.655+0000 7f048bfff700 1 -- 172.21.15.27:0/2072266074 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484011830 con 0x7f04940b5c90 2023-07-06T20:14:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f048bfff700 1 -- 172.21.15.27:0/2072266074 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0484011990 con 0x7f04940b5c90 2023-07-06T20:14:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f048bfff700 1 -- 172.21.15.27:0/2072266074 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484011d20 con 0x7f04940b5c90 2023-07-06T20:14:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 -- 172.21.15.27:0/2072266074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04940b5c90 msgr2=0x7f04940d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2072266074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04940b5c90 0x7f04940d2d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0484009840 tx=0x7f0484009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 -- 172.21.15.27:0/2072266074 shutdown_connections 2023-07-06T20:14:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2072266074 >> v2:172.21.15.153:3300/0 conn(0x7f04940b60e0 0x7f04940d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2072266074 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04940b5c90 0x7f04940d2d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 -- 172.21.15.27:0/2072266074 >> 172.21.15.27:0/2072266074 conn(0x7f049406c150 msgr2=0x7f049406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 -- 172.21.15.27:0/2072266074 shutdown_connections 2023-07-06T20:14:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.656+0000 7f049b9ff700 1 -- 172.21.15.27:0/2072266074 wait complete. 2023-07-06T20:14:50.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.658+0000 7f049b9ff700 1 Processor -- start 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.658+0000 7f049b9ff700 1 -- start start 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f049b9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04940b5c90 0x7f049415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f049b9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 0x7f049415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f049b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f049415fca0 0x7f04940b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f049b9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0494155f60 con 0x7f04940b60e0 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f049b9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049415c750 con 0x7f04940b5c90 2023-07-06T20:14:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f049b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049415c9b0 con 0x7f049415fca0 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f0498f9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 0x7f049415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f0498f9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 0x7f049415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46600/0 (socket says 172.21.15.27:46600) 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.659+0000 7f0498f9a700 1 -- 172.21.15.27:0/2115216935 learned_addr learned my addr 172.21.15.27:0/2115216935 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f049979b700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04940b5c90 0x7f049415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0499f9c700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f049415fca0 0x7f04940b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0498f9a700 1 -- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f049415fca0 msgr2=0x7f04940b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0498f9a700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f049415fca0 0x7f04940b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0498f9a700 1 -- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04940b5c90 msgr2=0x7f049415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0498f9a700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04940b5c90 0x7f049415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0498f9a700 1 -- 172.21.15.27:0/2115216935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840094a0 con 0x7f04940b60e0 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.660+0000 7f0498f9a700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 0x7f049415c210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04840090b0 tx=0x7f0484004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.661+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484004590 con 0x7f04940b60e0 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.661+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048401f070 con 0x7f04940b60e0 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.661+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049415cc90 con 0x7f04940b60e0 2023-07-06T20:14:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.661+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840038e0 con 0x7f04940b60e0 2023-07-06T20:14:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.661+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049415d250 con 0x7f04940b60e0 2023-07-06T20:14:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.662+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04941560d0 con 0x7f04940b60e0 2023-07-06T20:14:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.663+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0484003a40 con 0x7f04940b60e0 2023-07-06T20:14:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.664+0000 7f0489ffb700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f047806ace0 0x7f047806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.664+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0484021070 con 0x7f04940b60e0 2023-07-06T20:14:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.665+0000 7f049979b700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f047806ace0 0x7f047806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.665+0000 7f049979b700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f047806ace0 0x7f047806d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0490007fd0 tx=0x7f0490007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.668+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0484055d00 con 0x7f04940b60e0 2023-07-06T20:14:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.873+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0494155d90 con 0x7f04940b60e0 2023-07-06T20:14:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.874+0000 7f0489ffb700 1 -- 172.21.15.27:0/2115216935 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0484059320 con 0x7f04940b60e0 2023-07-06T20:14:50.876 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f047806ace0 msgr2=0x7f047806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f047806ace0 0x7f047806d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0490007fd0 tx=0x7f0490007dc0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 msgr2=0x7f049415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 0x7f049415c210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04840090b0 tx=0x7f0484004750 comp rx=0 tx=0).stop 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 shutdown_connections 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f047806ace0 0x7f047806d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f049415fca0 0x7f04940b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04940b5c90 0x7f049415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 --2- 172.21.15.27:0/2115216935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04940b60e0 0x7f049415c210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 >> 172.21.15.27:0/2115216935 conn(0x7f049406c150 msgr2=0x7f049415ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 shutdown_connections 2023-07-06T20:14:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:50.876+0000 7f049b9ff700 1 -- 172.21.15.27:0/2115216935 wait complete. 2023-07-06T20:14:50.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:51.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.102+0000 7fba5dfff700 1 Processor -- start 2023-07-06T20:14:51.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.102+0000 7fba5dfff700 1 -- start start 2023-07-06T20:14:51.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.103+0000 7fba5dfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5815fca0 0x7fba580d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.103+0000 7fba5dfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba580b5c90 0x7fba580d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.103+0000 7fba5dfff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba580cf780 con 0x7fba580b6100 2023-07-06T20:14:51.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.103+0000 7fba5dfff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580cd490 con 0x7fba580b5c90 2023-07-06T20:14:51.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.103+0000 7fba5dfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba580cd770 con 0x7fba5815fca0 2023-07-06T20:14:51.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.103+0000 7fba57fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fba580b6100 0x7fba580cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56770/0 (socket says 172.21.15.27:56770) 2023-07-06T20:14:51.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba57fff700 1 -- 172.21.15.27:0/318000971 learned_addr learned my addr 172.21.15.27:0/318000971 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:51.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba577fe700 1 --2- 172.21.15.27:0/318000971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5815fca0 0x7fba580d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba56ffd700 1 --2- 172.21.15.27:0/318000971 >> v2:172.21.15.153:3300/0 conn(0x7fba580b5c90 0x7fba580d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba567fc700 1 -- 172.21.15.27:0/318000971 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51774657 0 0) 0x7fba580cf780 con 0x7fba580b6100 2023-07-06T20:14:51.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba567fc700 1 -- 172.21.15.27:0/318000971 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba400035b0 con 0x7fba580b6100 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba577fe700 1 -- 172.21.15.27:0/318000971 >> v2:172.21.15.153:3300/0 conn(0x7fba580b5c90 msgr2=0x7fba580d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba577fe700 1 --2- 172.21.15.27:0/318000971 >> v2:172.21.15.153:3300/0 conn(0x7fba580b5c90 0x7fba580d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba577fe700 1 -- 172.21.15.27:0/318000971 >> v1:172.21.15.27:6789/0 conn(0x7fba580b6100 legacy=0x7fba580cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba577fe700 1 -- 172.21.15.27:0/318000971 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580cda50 con 0x7fba5815fca0 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.104+0000 7fba56ffd700 1 --2- 172.21.15.27:0/318000971 >> v2:172.21.15.153:3300/0 conn(0x7fba580b5c90 0x7fba580d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba577fe700 1 --2- 172.21.15.27:0/318000971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5815fca0 0x7fba580d1420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fba48009840 tx=0x7fba48009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e68ccb94b1077be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:51.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba567fc700 1 -- 172.21.15.27:0/318000971 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48011830 con 0x7fba5815fca0 2023-07-06T20:14:51.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba567fc700 1 -- 172.21.15.27:0/318000971 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba48011990 con 0x7fba5815fca0 2023-07-06T20:14:51.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba567fc700 1 -- 172.21.15.27:0/318000971 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48011d20 con 0x7fba5815fca0 2023-07-06T20:14:51.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba5dfff700 1 -- 172.21.15.27:0/318000971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5815fca0 msgr2=0x7fba580d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba5dfff700 1 --2- 172.21.15.27:0/318000971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5815fca0 0x7fba580d1420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fba48009840 tx=0x7fba48009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba5dfff700 1 -- 172.21.15.27:0/318000971 shutdown_connections 2023-07-06T20:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba5dfff700 1 --2- 172.21.15.27:0/318000971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba5815fca0 0x7fba580d1420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba5dfff700 1 --2- 172.21.15.27:0/318000971 >> v2:172.21.15.153:3300/0 conn(0x7fba580b5c90 0x7fba580d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.105+0000 7fba5dfff700 1 -- 172.21.15.27:0/318000971 >> 172.21.15.27:0/318000971 conn(0x7fba5806c150 msgr2=0x7fba5806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.106+0000 7fba5dfff700 1 -- 172.21.15.27:0/318000971 shutdown_connections 2023-07-06T20:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.106+0000 7fba5dfff700 1 -- 172.21.15.27:0/318000971 wait complete. 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.107+0000 7fba5dfff700 1 Processor -- start 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.107+0000 7fba5dfff700 1 -- start start 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.108+0000 7fba5dfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba580b5c90 0x7fba580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.108+0000 7fba5dfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba580b6100 0x7fba58153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.108+0000 7fba5dfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 0x7fba580cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.108+0000 7fba5dfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba580b52d0 con 0x7fba5815fca0 2023-07-06T20:14:51.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba5dfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba58154530 con 0x7fba580b6100 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba5dfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba58154810 con 0x7fba580b5c90 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 0x7fba580cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 0x7fba580cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46606/0 (socket says 172.21.15.27:46606) 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 -- 172.21.15.27:0/2910518987 learned_addr learned my addr 172.21.15.27:0/2910518987 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba577fe700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba580b5c90 0x7fba580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba56ffd700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba580b6100 0x7fba58153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 -- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba580b5c90 msgr2=0x7fba580b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba580b5c90 0x7fba580b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 -- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba580b6100 msgr2=0x7fba58153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba580b6100 0x7fba58153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 -- 172.21.15.27:0/2910518987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0094a0 con 0x7fba5815fca0 2023-07-06T20:14:51.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.109+0000 7fba57fff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 0x7fba580cb7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba5000cbb0 tx=0x7fba5000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:51.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.110+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000ab50 con 0x7fba5815fca0 2023-07-06T20:14:51.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.110+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5000acb0 con 0x7fba5815fca0 2023-07-06T20:14:51.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.110+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50012600 con 0x7fba5815fca0 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.110+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba480094a0 con 0x7fba5815fca0 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.110+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba58154dd0 con 0x7fba5815fca0 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.111+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba580008d0 con 0x7fba5815fca0 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.112+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba50012760 con 0x7fba5815fca0 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.113+0000 7fba3ffff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba4406ace0 0x7fba4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.113+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fba50089150 con 0x7fba5815fca0 2023-07-06T20:14:51.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.113+0000 7fba577fe700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba4406ace0 0x7fba4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.114+0000 7fba577fe700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba4406ace0 0x7fba4406d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba48000c00 tx=0x7fba48008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:51.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.118+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba50054270 con 0x7fba5815fca0 2023-07-06T20:14:51.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.322+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba580d2410 con 0x7fba5815fca0 2023-07-06T20:14:51.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.323+0000 7fba3ffff700 1 -- 172.21.15.27:0/2910518987 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba50057890 con 0x7fba5815fca0 2023-07-06T20:14:51.325 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:51.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba4406ace0 msgr2=0x7fba4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba4406ace0 0x7fba4406d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba48000c00 tx=0x7fba48008030 comp rx=0 tx=0).stop 2023-07-06T20:14:51.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 msgr2=0x7fba580cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 0x7fba580cb7f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba5000cbb0 tx=0x7fba5000cf70 comp rx=0 tx=0).stop 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 shutdown_connections 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fba4406ace0 0x7fba4406d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba5815fca0 0x7fba580cb7f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fba580b5c90 0x7fba580b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 --2- 172.21.15.27:0/2910518987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba580b6100 0x7fba58153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 >> 172.21.15.27:0/2910518987 conn(0x7fba5806c150 msgr2=0x7fba580b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:51.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 shutdown_connections 2023-07-06T20:14:51.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.325+0000 7fba5dfff700 1 -- 172.21.15.27:0/2910518987 wait complete. 2023-07-06T20:14:51.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:51.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.549+0000 7f4267fff700 1 Processor -- start 2023-07-06T20:14:51.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.549+0000 7f4267fff700 1 -- start start 2023-07-06T20:14:51.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.550+0000 7f4267fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42600b6110 0x7f42600cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.550+0000 7f4267fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42600cf780 con 0x7f42600b5c90 2023-07-06T20:14:51.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.550+0000 7f4267fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42600cd490 con 0x7f426015fca0 2023-07-06T20:14:51.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.550+0000 7f4267fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600cd770 con 0x7f42600b6110 2023-07-06T20:14:51.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426559a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f42600b5c90 0x7f42600d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56776/0 (socket says 172.21.15.27:56776) 2023-07-06T20:14:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426559a700 1 -- 172.21.15.27:0/735650988 learned_addr learned my addr 172.21.15.27:0/735650988 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426659c700 1 --2- 172.21.15.27:0/735650988 >> v2:172.21.15.27:3300/0 conn(0x7f42600b6110 0x7f42600cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426659c700 1 -- 172.21.15.27:0/735650988 >> v1:172.21.15.153:6789/0 conn(0x7f426015fca0 legacy=0x7f42600d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f4264d99700 1 -- 172.21.15.27:0/735650988 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487195122 0 0) 0x7f42600cf780 con 0x7f42600b5c90 2023-07-06T20:14:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426659c700 1 -- 172.21.15.27:0/735650988 >> v1:172.21.15.27:6789/0 conn(0x7f42600b5c90 legacy=0x7f42600d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426659c700 1 -- 172.21.15.27:0/735650988 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600cda50 con 0x7f42600b6110 2023-07-06T20:14:51.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.551+0000 7f426659c700 1 --2- 172.21.15.27:0/735650988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cfc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4258009d90 tx=0x7f425800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a86b900db9ec337 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4264d99700 1 -- 172.21.15.27:0/735650988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800e070 con 0x7f42600b6110 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4264d99700 1 -- 172.21.15.27:0/735650988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800bbd0 con 0x7f42600b6110 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4264d99700 1 -- 172.21.15.27:0/735650988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258012430 con 0x7f42600b6110 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 -- 172.21.15.27:0/735650988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 msgr2=0x7f42600cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 --2- 172.21.15.27:0/735650988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cfc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4258009d90 tx=0x7f425800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 -- 172.21.15.27:0/735650988 shutdown_connections 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 --2- 172.21.15.27:0/735650988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cfc10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 -- 172.21.15.27:0/735650988 >> 172.21.15.27:0/735650988 conn(0x7f426006c150 msgr2=0x7f426006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:51.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 -- 172.21.15.27:0/735650988 shutdown_connections 2023-07-06T20:14:51.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.552+0000 7f4267fff700 1 -- 172.21.15.27:0/735650988 wait complete. 2023-07-06T20:14:51.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.555+0000 7f4267fff700 1 Processor -- start 2023-07-06T20:14:51.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.555+0000 7f4267fff700 1 -- start start 2023-07-06T20:14:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f4267fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42600b5c90 0x7f42600cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f4267fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f4267fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f426015fca0 0x7f42600c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f4267fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260156db0 con 0x7f42600b6110 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f4267fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c76e0 con 0x7f42600b5c90 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f4267fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600cba30 con 0x7f426015fca0 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f426559a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.556+0000 7f426559a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46624/0 (socket says 172.21.15.27:46624) 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 -- 172.21.15.27:0/2217938115 learned_addr learned my addr 172.21.15.27:0/2217938115 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f4265d9b700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42600b5c90 0x7f42600cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426659c700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f426015fca0 0x7f42600c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 -- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f426015fca0 msgr2=0x7f42600c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f426015fca0 0x7f42600c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 -- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42600b5c90 msgr2=0x7f42600cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42600b5c90 0x7f42600cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 -- 172.21.15.27:0/2217938115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009a70 con 0x7f42600b6110 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f4265d9b700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42600b5c90 0x7f42600cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f426559a700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cb4f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f425000c550 tx=0x7f425000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.557+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250004c80 con 0x7f42600b6110 2023-07-06T20:14:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.558+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4250004e00 con 0x7f42600b6110 2023-07-06T20:14:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.558+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250007310 con 0x7f42600b6110 2023-07-06T20:14:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.558+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42600c6ad0 con 0x7f42600b6110 2023-07-06T20:14:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.558+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42600c7090 con 0x7f42600b6110 2023-07-06T20:14:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.559+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42600cec70 con 0x7f42600b6110 2023-07-06T20:14:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.560+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4250007470 con 0x7f42600b6110 2023-07-06T20:14:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.561+0000 7f42567fc700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f424406ad10 0x7f424406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.561+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4250089270 con 0x7f42600b6110 2023-07-06T20:14:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.561+0000 7f4265d9b700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f424406ad10 0x7f424406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.562+0000 7f4265d9b700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f424406ad10 0x7f424406d1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4260157a90 tx=0x7f425c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.565+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f42500542d0 con 0x7f42600b6110 2023-07-06T20:14:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.770+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42600c7850 con 0x7f42600b6110 2023-07-06T20:14:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.771+0000 7f42567fc700 1 -- 172.21.15.27:0/2217938115 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42500578f0 con 0x7f42600b6110 2023-07-06T20:14:51.773 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f424406ad10 msgr2=0x7f424406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f424406ad10 0x7f424406d1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4260157a90 tx=0x7f425c009040 comp rx=0 tx=0).stop 2023-07-06T20:14:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 msgr2=0x7f42600cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cb4f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f425000c550 tx=0x7f425000c910 comp rx=0 tx=0).stop 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 shutdown_connections 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f424406ad10 0x7f424406d1c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f426015fca0 0x7f42600c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42600b5c90 0x7f42600cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 --2- 172.21.15.27:0/2217938115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42600b6110 0x7f42600cb4f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 >> 172.21.15.27:0/2217938115 conn(0x7f426006c150 msgr2=0x7f42600b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 shutdown_connections 2023-07-06T20:14:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.773+0000 7f4267fff700 1 -- 172.21.15.27:0/2217938115 wait complete. 2023-07-06T20:14:51.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:52.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.999+0000 7f7ca97ff700 1 Processor -- start 2023-07-06T20:14:52.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:51.999+0000 7f7ca97ff700 1 -- start start 2023-07-06T20:14:52.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.000+0000 7f7ca97ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ca415d8b0 0x7f7ca40d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca97ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ca415dc80 0x7f7ca40d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca97ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca406cb60 con 0x7f7ca4160640 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca97ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca4161190 con 0x7f7ca415d8b0 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca97ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca41613a0 con 0x7f7ca415dc80 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ca415d8b0 0x7f7ca40d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ca415d8b0 0x7f7ca40d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59228/0 (socket says 172.21.15.27:59228) 2023-07-06T20:14:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2057891423 learned_addr learned my addr 172.21.15.27:0/2057891423 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:52.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7c9bfff700 1 --2- 172.21.15.27:0/2057891423 >> v2:172.21.15.153:3300/0 conn(0x7f7ca415dc80 0x7f7ca40d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2057891423 >> v2:172.21.15.153:3300/0 conn(0x7f7ca415dc80 msgr2=0x7f7ca40d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 --2- 172.21.15.27:0/2057891423 >> v2:172.21.15.153:3300/0 conn(0x7f7ca415dc80 0x7f7ca40d1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2057891423 >> v1:172.21.15.27:6789/0 conn(0x7f7ca4160640 legacy=0x7f7ca4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.001+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2057891423 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca406c230 con 0x7f7ca415d8b0 2023-07-06T20:14:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.002+0000 7f7ca2ffd700 1 --2- 172.21.15.27:0/2057891423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca40d0b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c940098f0 tx=0x7f7c94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd9b6fc3d9393d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.002+0000 7f7ca27fc700 1 -- 172.21.15.27:0/2057891423 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400b860 con 0x7f7ca415d8b0 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.002+0000 7f7ca27fc700 1 -- 172.21.15.27:0/2057891423 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9400b9c0 con 0x7f7ca415d8b0 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.002+0000 7f7ca27fc700 1 -- 172.21.15.27:0/2057891423 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400bd50 con 0x7f7ca415d8b0 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.002+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2057891423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 msgr2=0x7f7ca40d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.002+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2057891423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca40d0b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c940098f0 tx=0x7f7c94009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.003+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2057891423 shutdown_connections 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.003+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2057891423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca40d0b40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.003+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2057891423 >> v2:172.21.15.153:3300/0 conn(0x7f7ca415dc80 0x7f7ca40d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.003+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2057891423 >> 172.21.15.27:0/2057891423 conn(0x7f7ca406db20 msgr2=0x7f7ca406df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.003+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2057891423 shutdown_connections 2023-07-06T20:14:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.003+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2057891423 wait complete. 2023-07-06T20:14:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.004+0000 7f7ca97ff700 1 Processor -- start 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.004+0000 7f7ca97ff700 1 -- start start 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.005+0000 7f7ca97ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca4145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.005+0000 7f7ca97ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ca4160640 0x7f7ca41e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.005+0000 7f7ca97ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca4152e70 0x7f7ca40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.005+0000 7f7ca97ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4146340 con 0x7f7ca415d8b0 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.005+0000 7f7ca97ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41e6de0 con 0x7f7ca4152e70 2023-07-06T20:14:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.005+0000 7f7ca97ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41e70c0 con 0x7f7ca4160640 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca4145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca4145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59240/0 (socket says 172.21.15.27:59240) 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2495928314 learned_addr learned my addr 172.21.15.27:0/2495928314 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7c9bfff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ca4160640 0x7f7ca41e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca37fe700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca4152e70 0x7f7ca40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ca4160640 msgr2=0x7f7ca41e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ca4160640 0x7f7ca41e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca4152e70 msgr2=0x7f7ca40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca4152e70 0x7f7ca40ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 -- 172.21.15.27:0/2495928314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94009550 con 0x7f7ca415d8b0 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.006+0000 7f7ca2ffd700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca4145e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009c40 tx=0x7f7c94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.007+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94004300 con 0x7f7ca415d8b0 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.007+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9401f070 con 0x7f7ca415d8b0 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.007+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca41e73a0 con 0x7f7ca415d8b0 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.007+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940036a0 con 0x7f7ca415d8b0 2023-07-06T20:14:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.007+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca40b4db0 con 0x7f7ca415d8b0 2023-07-06T20:14:52.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.008+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca40008d0 con 0x7f7ca415d8b0 2023-07-06T20:14:52.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.009+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c94021030 con 0x7f7ca415d8b0 2023-07-06T20:14:52.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.010+0000 7f7c9b7fe700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7c8406ace0 0x7f7c8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.010+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7c9408a810 con 0x7f7ca415d8b0 2023-07-06T20:14:52.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.010+0000 7f7c9bfff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7c8406ace0 0x7f7c8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.011+0000 7f7c9bfff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7c8406ace0 0x7f7c8406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4153760 tx=0x7f7c8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:52.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.014+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7c94055920 con 0x7f7ca415d8b0 2023-07-06T20:14:52.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.220+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ca40d1a20 con 0x7f7ca415d8b0 2023-07-06T20:14:52.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.220+0000 7f7c9b7fe700 1 -- 172.21.15.27:0/2495928314 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c94058f40 con 0x7f7ca415d8b0 2023-07-06T20:14:52.222 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:52.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7c8406ace0 msgr2=0x7f7c8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7c8406ace0 0x7f7c8406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4153760 tx=0x7f7c8c008040 comp rx=0 tx=0).stop 2023-07-06T20:14:52.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 msgr2=0x7f7ca4145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca4145e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009c40 tx=0x7f7c94004750 comp rx=0 tx=0).stop 2023-07-06T20:14:52.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 shutdown_connections 2023-07-06T20:14:52.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7c8406ace0 0x7f7c8406d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ca4152e70 0x7f7ca40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ca415d8b0 0x7f7ca4145e00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 --2- 172.21.15.27:0/2495928314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7ca4160640 0x7f7ca41e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 >> 172.21.15.27:0/2495928314 conn(0x7f7ca406db20 msgr2=0x7f7ca40b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 shutdown_connections 2023-07-06T20:14:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.223+0000 7f7ca97ff700 1 -- 172.21.15.27:0/2495928314 wait complete. 2023-07-06T20:14:52.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:14:52.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.447+0000 7f791e1ff700 1 Processor -- start 2023-07-06T20:14:52.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.448+0000 7f791e1ff700 1 -- start start 2023-07-06T20:14:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.449+0000 7f791e1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7918158cd0 0x7f79180cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f791e1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79180cd370 con 0x7f79180c9010 2023-07-06T20:14:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f791e1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79180ad7f0 con 0x7f79180ce420 2023-07-06T20:14:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f791e1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180cbf50 con 0x7f7918158cd0 2023-07-06T20:14:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79177fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7918158cd0 0x7f79180cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79177fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7918158cd0 0x7f79180cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59244/0 (socket says 172.21.15.27:59244) 2023-07-06T20:14:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79177fe700 1 -- 172.21.15.27:0/709407714 learned_addr learned my addr 172.21.15.27:0/709407714 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f7916ffd700 1 --1- 172.21.15.27:0/709407714 >> v1:172.21.15.27:6789/0 conn(0x7f79180c9010 0x7f791806b220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45334/0 (socket says 172.21.15.27:45334) 2023-07-06T20:14:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79177fe700 1 -- 172.21.15.27:0/709407714 >> v1:172.21.15.153:6789/0 conn(0x7f79180ce420 legacy=0x7f79180cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79177fe700 1 -- 172.21.15.27:0/709407714 >> v1:172.21.15.27:6789/0 conn(0x7f79180c9010 legacy=0x7f791806b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79177fe700 1 -- 172.21.15.27:0/709407714 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180cc160 con 0x7f7918158cd0 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.450+0000 7f79167fc700 1 -- 172.21.15.27:0/709407714 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453419921 0 0) 0x7f79180cd370 con 0x7f79180c9010 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.451+0000 7f79177fe700 1 --2- 172.21.15.27:0/709407714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7918158cd0 0x7f79180cdee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f79080098f0 tx=0x7f7908009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedff24ce5bcd5f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.451+0000 7f79167fc700 1 -- 172.21.15.27:0/709407714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800b8a0 con 0x7f7918158cd0 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.451+0000 7f79167fc700 1 -- 172.21.15.27:0/709407714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790800ba00 con 0x7f7918158cd0 2023-07-06T20:14:52.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.451+0000 7f79167fc700 1 -- 172.21.15.27:0/709407714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800bd90 con 0x7f7918158cd0 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.451+0000 7f791e1ff700 1 -- 172.21.15.27:0/709407714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7918158cd0 msgr2=0x7f79180cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.451+0000 7f791e1ff700 1 --2- 172.21.15.27:0/709407714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7918158cd0 0x7f79180cdee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f79080098f0 tx=0x7f7908009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.452+0000 7f791e1ff700 1 -- 172.21.15.27:0/709407714 shutdown_connections 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.452+0000 7f791e1ff700 1 --2- 172.21.15.27:0/709407714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7918158cd0 0x7f79180cdee0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.452+0000 7f791e1ff700 1 -- 172.21.15.27:0/709407714 >> 172.21.15.27:0/709407714 conn(0x7f79180cb440 msgr2=0x7f79180cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.452+0000 7f791e1ff700 1 -- 172.21.15.27:0/709407714 shutdown_connections 2023-07-06T20:14:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.452+0000 7f791e1ff700 1 -- 172.21.15.27:0/709407714 wait complete. 2023-07-06T20:14:52.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.453+0000 7f791e1ff700 1 Processor -- start 2023-07-06T20:14:52.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.454+0000 7f791e1ff700 1 -- start start 2023-07-06T20:14:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f791e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79180c9010 0x7f79180b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f791e1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 0x7f79180c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f791e1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7918158cd0 0x7f791806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f791e1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791814b280 con 0x7f79180ce420 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f791e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c38d0 con 0x7f79180c9010 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f791e1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c3bb0 con 0x7f7918158cd0 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 0x7f79180c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 0x7f79180c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59250/0 (socket says 172.21.15.27:59250) 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 -- 172.21.15.27:0/1655667062 learned_addr learned my addr 172.21.15.27:0/1655667062 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f79177fe700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79180c9010 0x7f79180b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7917fff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7918158cd0 0x7f791806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 -- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7918158cd0 msgr2=0x7f791806c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7918158cd0 0x7f791806c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 -- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79180c9010 msgr2=0x7f79180b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79180c9010 0x7f79180b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f7916ffd700 1 -- 172.21.15.27:0/1655667062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908009550 con 0x7f79180ce420 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.455+0000 7f79177fe700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79180c9010 0x7f79180b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.456+0000 7f7916ffd700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 0x7f79180c5a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f790c00c520 tx=0x7f790c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.456+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c004c80 con 0x7f79180ce420 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.456+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c004e20 con 0x7f79180ce420 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.456+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79180c3e90 con 0x7f79180ce420 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.456+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c007310 con 0x7f79180ce420 2023-07-06T20:14:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.456+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79180c4450 con 0x7f79180ce420 2023-07-06T20:14:52.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.457+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918071390 con 0x7f79180ce420 2023-07-06T20:14:52.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.458+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f790c007470 con 0x7f79180ce420 2023-07-06T20:14:52.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.460+0000 7f78fffff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f790406ace0 0x7f790406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:52.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.460+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f790c0892b0 con 0x7f79180ce420 2023-07-06T20:14:52.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.460+0000 7f79177fe700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f790406ace0 0x7f790406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:52.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.461+0000 7f79177fe700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f790406ace0 0x7f790406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7908009130 tx=0x7f790801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:52.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.464+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f790c054280 con 0x7f79180ce420 2023-07-06T20:14:52.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.690+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79180c4640 con 0x7f79180ce420 2023-07-06T20:14:52.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.690+0000 7f78fffff700 1 -- 172.21.15.27:0/1655667062 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f790c0578a0 con 0x7f79180ce420 2023-07-06T20:14:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f790406ace0 msgr2=0x7f790406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f790406ace0 0x7f790406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7908009130 tx=0x7f790801c040 comp rx=0 tx=0).stop 2023-07-06T20:14:52.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 msgr2=0x7f79180c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:52.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 0x7f79180c5a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f790c00c520 tx=0x7f790c00c8e0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 shutdown_connections 2023-07-06T20:14:52.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f790406ace0 0x7f790406d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7918158cd0 0x7f791806c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79180c9010 0x7f79180b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 --2- 172.21.15.27:0/1655667062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79180ce420 0x7f79180c5a90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 >> 172.21.15.27:0/1655667062 conn(0x7f79180cb440 msgr2=0x7f79180b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 shutdown_connections 2023-07-06T20:14:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:52.693+0000 7f791e1ff700 1 -- 172.21.15.27:0/1655667062 wait complete. 2023-07-06T20:14:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-07-06T20:14:52.712 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-06T20:14:52.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:52.713 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-07-06T20:14:52.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:14:52.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:52.806 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T20:14:52.875 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:52.875 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:52.922 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:52.922 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:53.008 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:14:53.009 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:53.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:53.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.335+0000 7f2b531ff700 1 Processor -- start 2023-07-06T20:14:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.336+0000 7f2b531ff700 1 -- start start 2023-07-06T20:14:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b531ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b4c06c330 0x7f2b4c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b531ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2b4c0c9010 0x7f2b4c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b531ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4c06cc40 con 0x7f2b4c1565f0 2023-07-06T20:14:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b531ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b4c0c9a60 con 0x7f2b4c06c330 2023-07-06T20:14:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b531ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b4c159b70 con 0x7f2b4c0c9010 2023-07-06T20:14:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b50f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b4c06c330 0x7f2b4c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b50f9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b4c06c330 0x7f2b4c06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59252/0 (socket says 172.21.15.27:59252) 2023-07-06T20:14:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b50f9b700 1 -- 172.21.15.27:0/792488225 learned_addr learned my addr 172.21.15.27:0/792488225 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.337+0000 7f2b4bfff700 1 --2- 172.21.15.27:0/792488225 >> v2:172.21.15.153:3300/0 conn(0x7f2b4c0c9010 0x7f2b4c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b50f9b700 1 -- 172.21.15.27:0/792488225 >> v2:172.21.15.153:3300/0 conn(0x7f2b4c0c9010 msgr2=0x7f2b4c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b50f9b700 1 --2- 172.21.15.27:0/792488225 >> v2:172.21.15.153:3300/0 conn(0x7f2b4c0c9010 0x7f2b4c0c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b50f9b700 1 -- 172.21.15.27:0/792488225 >> v1:172.21.15.153:6789/0 conn(0x7f2b4c1565f0 legacy=0x7f2b4c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b50f9b700 1 -- 172.21.15.27:0/792488225 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c159e50 con 0x7f2b4c06c330 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b50f9b700 1 --2- 172.21.15.27:0/792488225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c06c330 0x7f2b4c06c700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c06f4d0 tx=0x7f2b40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13f82191bf118d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b4b7fe700 1 -- 172.21.15.27:0/792488225 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000b870 con 0x7f2b4c06c330 2023-07-06T20:14:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b4b7fe700 1 -- 172.21.15.27:0/792488225 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4000b9d0 con 0x7f2b4c06c330 2023-07-06T20:14:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.338+0000 7f2b4b7fe700 1 -- 172.21.15.27:0/792488225 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000bd60 con 0x7f2b4c06c330 2023-07-06T20:14:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 -- 172.21.15.27:0/792488225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c06c330 msgr2=0x7f2b4c06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 --2- 172.21.15.27:0/792488225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c06c330 0x7f2b4c06c700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c06f4d0 tx=0x7f2b40009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 -- 172.21.15.27:0/792488225 shutdown_connections 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 --2- 172.21.15.27:0/792488225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c06c330 0x7f2b4c06c700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 --2- 172.21.15.27:0/792488225 >> v2:172.21.15.153:3300/0 conn(0x7f2b4c0c9010 0x7f2b4c0c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 -- 172.21.15.27:0/792488225 >> 172.21.15.27:0/792488225 conn(0x7f2b4c0cb440 msgr2=0x7f2b4c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 -- 172.21.15.27:0/792488225 shutdown_connections 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.339+0000 7f2b531ff700 1 -- 172.21.15.27:0/792488225 wait complete. 2023-07-06T20:14:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.341+0000 7f2b531ff700 1 Processor -- start 2023-07-06T20:14:53.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.341+0000 7f2b531ff700 1 -- start start 2023-07-06T20:14:53.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b531ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b4c06c330 0x7f2b4c1481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b531ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b4c0c9010 0x7f2b4c14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b531ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 0x7f2b4c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b531ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0c5a90 con 0x7f2b4c1565f0 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b531ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c14b970 con 0x7f2b4c06c330 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b531ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0aed80 con 0x7f2b4c0c9010 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 0x7f2b4c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 0x7f2b4c14b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59260/0 (socket says 172.21.15.27:59260) 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 -- 172.21.15.27:0/2310014540 learned_addr learned my addr 172.21.15.27:0/2310014540 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b50f9b700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b4c06c330 0x7f2b4c1481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b4bfff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b4c0c9010 0x7f2b4c14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 -- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b4c0c9010 msgr2=0x7f2b4c14ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b4c0c9010 0x7f2b4c14ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 -- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b4c06c330 msgr2=0x7f2b4c1481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b4c06c330 0x7f2b4c1481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b5179c700 1 -- 172.21.15.27:0/2310014540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40009550 con 0x7f2b4c1565f0 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.342+0000 7f2b4bfff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b4c0c9010 0x7f2b4c14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.343+0000 7f2b5179c700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 0x7f2b4c14b160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400cc80 tx=0x7f2b4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.343+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44003c00 con 0x7f2b4c1565f0 2023-07-06T20:14:53.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.343+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b44003d60 con 0x7f2b4c1565f0 2023-07-06T20:14:53.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.343+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400ab50 con 0x7f2b4c1565f0 2023-07-06T20:14:53.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.343+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c14b6a0 con 0x7f2b4c1565f0 2023-07-06T20:14:53.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.343+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c0ae730 con 0x7f2b4c1565f0 2023-07-06T20:14:53.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.344+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4c0aef10 con 0x7f2b4c1565f0 2023-07-06T20:14:53.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.345+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b4400acb0 con 0x7f2b4c1565f0 2023-07-06T20:14:53.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.346+0000 7f2b497fa700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2b3006ace0 0x7f2b3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.347+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2b440895c0 con 0x7f2b4c1565f0 2023-07-06T20:14:53.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.347+0000 7f2b50f9b700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2b3006ace0 0x7f2b3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.348+0000 7f2b50f9b700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2b3006ace0 0x7f2b3006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b40014040 tx=0x7f2b40008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:53.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.350+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b44054590 con 0x7f2b4c1565f0 2023-07-06T20:14:53.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.556+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b4c0008d0 con 0x7f2b4c1565f0 2023-07-06T20:14:53.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.557+0000 7f2b497fa700 1 -- 172.21.15.27:0/2310014540 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f2b44057bb0 con 0x7f2b4c1565f0 2023-07-06T20:14:53.559 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:53.559 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:53.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2b3006ace0 msgr2=0x7f2b3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2b3006ace0 0x7f2b3006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b40014040 tx=0x7f2b40008030 comp rx=0 tx=0).stop 2023-07-06T20:14:53.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 msgr2=0x7f2b4c14b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 0x7f2b4c14b160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400cc80 tx=0x7f2b4400cf90 comp rx=0 tx=0).stop 2023-07-06T20:14:53.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 shutdown_connections 2023-07-06T20:14:53.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2b3006ace0 0x7f2b3006d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b4c1565f0 0x7f2b4c14b160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b4c06c330 0x7f2b4c1481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 --2- 172.21.15.27:0/2310014540 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2b4c0c9010 0x7f2b4c14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.559+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 >> 172.21.15.27:0/2310014540 conn(0x7f2b4c0cb440 msgr2=0x7f2b4c158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:53.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.560+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 shutdown_connections 2023-07-06T20:14:53.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.560+0000 7f2b531ff700 1 -- 172.21.15.27:0/2310014540 wait complete. 2023-07-06T20:14:53.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:53.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.785+0000 7f15997ff700 1 Processor -- start 2023-07-06T20:14:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.786+0000 7f15997ff700 1 -- start start 2023-07-06T20:14:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15997ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15940b0440 0x7f1594143900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15997ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f159415b0e0 0x7f159415b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15997ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15941441f0 con 0x7f159415df60 2023-07-06T20:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15997ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159415a930 con 0x7f159415b0e0 2023-07-06T20:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15997ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159415e3d0 con 0x7f15940b0440 2023-07-06T20:14:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15937fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f159415b0e0 0x7f159415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15937fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f159415b0e0 0x7f159415b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59266/0 (socket says 172.21.15.27:59266) 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f15937fe700 1 -- 172.21.15.27:0/1655114767 learned_addr learned my addr 172.21.15.27:0/1655114767 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.787+0000 7f1592ffd700 1 --2- 172.21.15.27:0/1655114767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15940b0440 0x7f1594143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15937fe700 1 -- 172.21.15.27:0/1655114767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15940b0440 msgr2=0x7f1594143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15937fe700 1 --2- 172.21.15.27:0/1655114767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15940b0440 0x7f1594143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15937fe700 1 -- 172.21.15.27:0/1655114767 >> v1:172.21.15.153:6789/0 conn(0x7f159415df60 legacy=0x7f15940b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15937fe700 1 -- 172.21.15.27:0/1655114767 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159415e660 con 0x7f159415b0e0 2023-07-06T20:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15937fe700 1 --2- 172.21.15.27:0/1655114767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f159415b0e0 0x7f159415b560 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f158c009e50 tx=0x7f158c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480cb17f52809c36 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15927fc700 1 -- 172.21.15.27:0/1655114767 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c00e070 con 0x7f159415b0e0 2023-07-06T20:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15927fc700 1 -- 172.21.15.27:0/1655114767 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c00b910 con 0x7f159415b0e0 2023-07-06T20:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.788+0000 7f15927fc700 1 -- 172.21.15.27:0/1655114767 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c00bcc0 con 0x7f159415b0e0 2023-07-06T20:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 -- 172.21.15.27:0/1655114767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f159415b0e0 msgr2=0x7f159415b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 --2- 172.21.15.27:0/1655114767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f159415b0e0 0x7f159415b560 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f158c009e50 tx=0x7f158c00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 -- 172.21.15.27:0/1655114767 shutdown_connections 2023-07-06T20:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 --2- 172.21.15.27:0/1655114767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f159415b0e0 0x7f159415b560 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 --2- 172.21.15.27:0/1655114767 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f15940b0440 0x7f1594143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 -- 172.21.15.27:0/1655114767 >> 172.21.15.27:0/1655114767 conn(0x7f15940cb440 msgr2=0x7f15940cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 -- 172.21.15.27:0/1655114767 shutdown_connections 2023-07-06T20:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.789+0000 7f15997ff700 1 -- 172.21.15.27:0/1655114767 wait complete. 2023-07-06T20:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.791+0000 7f15997ff700 1 Processor -- start 2023-07-06T20:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.791+0000 7f15997ff700 1 -- start start 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15997ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 0x7f15941573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15997ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f159415b0e0 0x7f15941e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15997ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f159415df60 0x7f15941e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15997ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15941f5c80 con 0x7f15940b0440 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15997ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15941f6a00 con 0x7f159415b0e0 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15997ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15941f5f20 con 0x7f159415df60 2023-07-06T20:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 0x7f15941573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 0x7f15941573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59280/0 (socket says 172.21.15.27:59280) 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f1592ffd700 1 -- 172.21.15.27:0/2599137775 learned_addr learned my addr 172.21.15.27:0/2599137775 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f158bfff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f159415b0e0 0x7f15941e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.792+0000 7f15937fe700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f159415df60 0x7f15941e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f1592ffd700 1 -- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f159415df60 msgr2=0x7f15941e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f1592ffd700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f159415df60 0x7f15941e51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f1592ffd700 1 -- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f159415b0e0 msgr2=0x7f15941e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f1592ffd700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f159415b0e0 0x7f15941e4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f1592ffd700 1 -- 172.21.15.27:0/2599137775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c009b30 con 0x7f15940b0440 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f158bfff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f159415b0e0 0x7f15941e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f1592ffd700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 0x7f15941573c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f158400e870 tx=0x7f158400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584009590 con 0x7f15940b0440 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400ee70 con 0x7f15940b0440 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15941f6200 con 0x7f15940b0440 2023-07-06T20:14:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400ca50 con 0x7f15940b0440 2023-07-06T20:14:53.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.793+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1594157900 con 0x7f15940b0440 2023-07-06T20:14:53.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.794+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159406fa50 con 0x7f15940b0440 2023-07-06T20:14:53.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.795+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f158400cbb0 con 0x7f15940b0440 2023-07-06T20:14:53.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.796+0000 7f158b7fe700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f157406ace0 0x7f157406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:53.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.796+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f15840894e0 con 0x7f15940b0440 2023-07-06T20:14:53.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.797+0000 7f158bfff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f157406ace0 0x7f157406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:53.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.797+0000 7f158bfff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f157406ace0 0x7f157406d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15941f5b40 tx=0x7f157c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:53.800+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1584057fd0 con 0x7f15940b0440 2023-07-06T20:14:54.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.005+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15940008d0 con 0x7f15940b0440 2023-07-06T20:14:54.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.005+0000 7f158b7fe700 1 -- 172.21.15.27:0/2599137775 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f158400ce60 con 0x7f15940b0440 2023-07-06T20:14:54.007 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:54.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f157406ace0 msgr2=0x7f157406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f157406ace0 0x7f157406d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15941f5b40 tx=0x7f157c009040 comp rx=0 tx=0).stop 2023-07-06T20:14:54.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 msgr2=0x7f15941573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 0x7f15941573c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f158400e870 tx=0x7f158400ec30 comp rx=0 tx=0).stop 2023-07-06T20:14:54.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 shutdown_connections 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f157406ace0 0x7f157406d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f159415df60 0x7f15941e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15940b0440 0x7f15941573c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 --2- 172.21.15.27:0/2599137775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f159415b0e0 0x7f15941e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 >> 172.21.15.27:0/2599137775 conn(0x7f15940cb440 msgr2=0x7f1594159590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 shutdown_connections 2023-07-06T20:14:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.008+0000 7f15997ff700 1 -- 172.21.15.27:0/2599137775 wait complete. 2023-07-06T20:14:54.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:54.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.234+0000 7f8c7f5ff700 1 Processor -- start 2023-07-06T20:14:54.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.234+0000 7f8c7f5ff700 1 -- start start 2023-07-06T20:14:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7f5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c780d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7f5ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8c780b6100 0x7f8c780cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7f5ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780cf780 con 0x7f8c780b5c90 2023-07-06T20:14:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7f5ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780cd490 con 0x7f8c780b6100 2023-07-06T20:14:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7f5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780cd770 con 0x7f8c7815fca0 2023-07-06T20:14:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7db9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8c780b6100 0x7f8c780cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7db9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8c780b6100 0x7f8c780cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59292/0 (socket says 172.21.15.27:59292) 2023-07-06T20:14:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7db9c700 1 -- 172.21.15.27:0/272028153 learned_addr learned my addr 172.21.15.27:0/272028153 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7d39b700 1 --2- 172.21.15.27:0/272028153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c780d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7db9c700 1 -- 172.21.15.27:0/272028153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 msgr2=0x7f8c780d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7db9c700 1 --2- 172.21.15.27:0/272028153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c780d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.236+0000 7f8c7db9c700 1 -- 172.21.15.27:0/272028153 >> v1:172.21.15.153:6789/0 conn(0x7f8c780b5c90 legacy=0x7f8c780d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.237+0000 7f8c7db9c700 1 -- 172.21.15.27:0/272028153 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780cda50 con 0x7f8c780b6100 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.237+0000 7f8c7db9c700 1 --2- 172.21.15.27:0/272028153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c780cfc10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009e50 tx=0x7f8c7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fed6f2e5a8d061 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.237+0000 7f8c6ffff700 1 -- 172.21.15.27:0/272028153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000e070 con 0x7f8c780b6100 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.237+0000 7f8c6ffff700 1 -- 172.21.15.27:0/272028153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7000b9c0 con 0x7f8c780b6100 2023-07-06T20:14:54.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.237+0000 7f8c6ffff700 1 -- 172.21.15.27:0/272028153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000bd70 con 0x7f8c780b6100 2023-07-06T20:14:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/272028153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 msgr2=0x7f8c780cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/272028153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c780cfc10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009e50 tx=0x7f8c7000b330 comp rx=0 tx=0).stop 2023-07-06T20:14:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/272028153 shutdown_connections 2023-07-06T20:14:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/272028153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c780cfc10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/272028153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c780d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/272028153 >> 172.21.15.27:0/272028153 conn(0x7f8c7806c150 msgr2=0x7f8c7806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/272028153 shutdown_connections 2023-07-06T20:14:54.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.238+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/272028153 wait complete. 2023-07-06T20:14:54.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.240+0000 7f8c7f5ff700 1 Processor -- start 2023-07-06T20:14:54.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.241+0000 7f8c7f5ff700 1 -- start start 2023-07-06T20:14:54.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7f5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c780b5c90 0x7f8c781f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7f5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c781572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7f5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c7815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7f5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7815b830 con 0x7f8c780b6100 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7f5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7815b090 con 0x7f8c780b5c90 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7f5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7815ba40 con 0x7f8c7815fca0 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c781572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c781572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59294/0 (socket says 172.21.15.27:59294) 2023-07-06T20:14:54.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 -- 172.21.15.27:0/243532316 learned_addr learned my addr 172.21.15.27:0/243532316 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7d39b700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c780b5c90 0x7f8c781f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7db9c700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c7815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 -- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 msgr2=0x7f8c7815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c7815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 -- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c780b5c90 msgr2=0x7f8c781f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c780b5c90 0x7f8c781f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.242+0000 7f8c7cb9a700 1 -- 172.21.15.27:0/243532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009b30 con 0x7f8c780b6100 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.243+0000 7f8c7cb9a700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c781572b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c4e0 tx=0x7f8c6800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.243+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680052d0 con 0x7f8c780b6100 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.243+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68005430 con 0x7f8c780b6100 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.243+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7815bd20 con 0x7f8c780b6100 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.243+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680038c0 con 0x7f8c780b6100 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.243+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7815c2e0 con 0x7f8c780b6100 2023-07-06T20:14:54.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.244+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7815c4d0 con 0x7f8c780b6100 2023-07-06T20:14:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.245+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c68003a20 con 0x7f8c780b6100 2023-07-06T20:14:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.246+0000 7f8c6dffb700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8c5c06ace0 0x7f8c5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.246+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8c680889e0 con 0x7f8c780b6100 2023-07-06T20:14:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.247+0000 7f8c7d39b700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8c5c06ace0 0x7f8c5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.247+0000 7f8c7d39b700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8c5c06ace0 0x7f8c5c06d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c780d21b0 tx=0x7f8c74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:54.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.251+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c68053a40 con 0x7f8c780b6100 2023-07-06T20:14:54.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.457+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8c780008d0 con 0x7f8c780b6100 2023-07-06T20:14:54.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.458+0000 7f8c6dffb700 1 -- 172.21.15.27:0/243532316 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c68017020 con 0x7f8c780b6100 2023-07-06T20:14:54.460 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:54.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8c5c06ace0 msgr2=0x7f8c5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8c5c06ace0 0x7f8c5c06d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c780d21b0 tx=0x7f8c74008040 comp rx=0 tx=0).stop 2023-07-06T20:14:54.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 msgr2=0x7f8c781572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c781572b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c4e0 tx=0x7f8c6800c8a0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 shutdown_connections 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8c5c06ace0 0x7f8c5c06d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8c7815fca0 0x7f8c7815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c780b5c90 0x7f8c781f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 --2- 172.21.15.27:0/243532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c780b6100 0x7f8c781572b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 >> 172.21.15.27:0/243532316 conn(0x7f8c7806c150 msgr2=0x7f8c780d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 shutdown_connections 2023-07-06T20:14:54.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.460+0000 7f8c7f5ff700 1 -- 172.21.15.27:0/243532316 wait complete. 2023-07-06T20:14:54.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.684+0000 7f06165ff700 1 Processor -- start 2023-07-06T20:14:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.684+0000 7f06165ff700 1 -- start start 2023-07-06T20:14:54.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.685+0000 7f06165ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f06100b5c90 0x7f06100d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.685+0000 7f06165ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06100cf780 con 0x7f06100b6110 2023-07-06T20:14:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.685+0000 7f06165ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06100cd490 con 0x7f061015fca0 2023-07-06T20:14:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.685+0000 7f06165ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100cd770 con 0x7f06100b5c90 2023-07-06T20:14:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f060f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f06100b5c90 0x7f06100d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f0614b9c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f06100b6110 0x7f06100cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45348/0 (socket says 172.21.15.27:45348) 2023-07-06T20:14:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f0614b9c700 1 -- 172.21.15.27:0/927199145 learned_addr learned my addr 172.21.15.27:0/927199145 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:54.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f060f7fe700 1 -- 172.21.15.27:0/927199145 >> v1:172.21.15.153:6789/0 conn(0x7f061015fca0 legacy=0x7f06100d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f060f7fe700 1 -- 172.21.15.27:0/927199145 >> v1:172.21.15.27:6789/0 conn(0x7f06100b6110 legacy=0x7f06100cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f060f7fe700 1 -- 172.21.15.27:0/927199145 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100cda50 con 0x7f06100b5c90 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f060effd700 1 -- 172.21.15.27:0/927199145 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141917112 0 0) 0x7f06100cf780 con 0x7f06100b6110 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.686+0000 7f060f7fe700 1 --2- 172.21.15.27:0/927199145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100d2d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0604009840 tx=0x7f0604009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1ea9e0bdceae44 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f060effd700 1 -- 172.21.15.27:0/927199145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604011830 con 0x7f06100b5c90 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f060effd700 1 -- 172.21.15.27:0/927199145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604011990 con 0x7f06100b5c90 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f060effd700 1 -- 172.21.15.27:0/927199145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604011d20 con 0x7f06100b5c90 2023-07-06T20:14:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f06165ff700 1 -- 172.21.15.27:0/927199145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 msgr2=0x7f06100d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f06165ff700 1 --2- 172.21.15.27:0/927199145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100d2d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0604009840 tx=0x7f0604009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:54.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f06165ff700 1 -- 172.21.15.27:0/927199145 shutdown_connections 2023-07-06T20:14:54.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f06165ff700 1 --2- 172.21.15.27:0/927199145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100d2d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f06165ff700 1 -- 172.21.15.27:0/927199145 >> 172.21.15.27:0/927199145 conn(0x7f061006c150 msgr2=0x7f061006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:54.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.687+0000 7f06165ff700 1 -- 172.21.15.27:0/927199145 shutdown_connections 2023-07-06T20:14:54.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.688+0000 7f06165ff700 1 -- 172.21.15.27:0/927199145 wait complete. 2023-07-06T20:14:54.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.689+0000 7f06165ff700 1 Processor -- start 2023-07-06T20:14:54.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.689+0000 7f06165ff700 1 -- start start 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.690+0000 7f06165ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.690+0000 7f06165ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06100b6110 0x7f06101f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.690+0000 7f06165ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f061015fca0 0x7f06101e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.690+0000 7f06165ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100cc270 con 0x7f06100b5c90 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.690+0000 7f06165ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100cc550 con 0x7f061015fca0 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f06165ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100cc830 con 0x7f06100b6110 2023-07-06T20:14:54.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59316/0 (socket says 172.21.15.27:59316) 2023-07-06T20:14:54.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 -- 172.21.15.27:0/4072136740 learned_addr learned my addr 172.21.15.27:0/4072136740 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:54.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f0614b9c700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f061015fca0 0x7f06101e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060f7fe700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06100b6110 0x7f06101f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 -- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06100b6110 msgr2=0x7f06101f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06100b6110 0x7f06101f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 -- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f061015fca0 msgr2=0x7f06101e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f061015fca0 0x7f06101e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 -- 172.21.15.27:0/4072136740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040094a0 con 0x7f06100b5c90 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060f7fe700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06100b6110 0x7f06101f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.691+0000 7f060ffff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100cb940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f060000c450 tx=0x7f060000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.692+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06000052d0 con 0x7f06100b5c90 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.692+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600005430 con 0x7f06100b5c90 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.692+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06000038c0 con 0x7f06100b5c90 2023-07-06T20:14:54.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.692+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06101460a0 con 0x7f06100b5c90 2023-07-06T20:14:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.692+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100cbe80 con 0x7f06100b5c90 2023-07-06T20:14:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.693+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100cc010 con 0x7f06100b5c90 2023-07-06T20:14:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.694+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0600003a20 con 0x7f06100b5c90 2023-07-06T20:14:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.695+0000 7f060cff9700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05fc06ace0 0x7f05fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.695+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f06000888a0 con 0x7f06100b5c90 2023-07-06T20:14:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.696+0000 7f060f7fe700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05fc06ace0 0x7f05fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.696+0000 7f060f7fe700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05fc06ace0 0x7f05fc06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06101590b0 tx=0x7f060401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.700+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0600053900 con 0x7f06100b5c90 2023-07-06T20:14:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.905+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06100008d0 con 0x7f06100b5c90 2023-07-06T20:14:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.906+0000 7f060cff9700 1 -- 172.21.15.27:0/4072136740 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0600056f20 con 0x7f06100b5c90 2023-07-06T20:14:54.908 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05fc06ace0 msgr2=0x7f05fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05fc06ace0 0x7f05fc06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06101590b0 tx=0x7f060401c040 comp rx=0 tx=0).stop 2023-07-06T20:14:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 msgr2=0x7f06100cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100cb940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f060000c450 tx=0x7f060000c760 comp rx=0 tx=0).stop 2023-07-06T20:14:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 shutdown_connections 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f05fc06ace0 0x7f05fc06d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f061015fca0 0x7f06101e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06100b5c90 0x7f06100cb940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 --2- 172.21.15.27:0/4072136740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06100b6110 0x7f06101f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 >> 172.21.15.27:0/4072136740 conn(0x7f061006c150 msgr2=0x7f061015a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 shutdown_connections 2023-07-06T20:14:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:54.908+0000 7f06165ff700 1 -- 172.21.15.27:0/4072136740 wait complete. 2023-07-06T20:14:54.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.137+0000 7f6df67ff700 1 Processor -- start 2023-07-06T20:14:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.137+0000 7f6df67ff700 1 -- start start 2023-07-06T20:14:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6df67ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6df00b6110 0x7f6df00cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6df67ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df00cf780 con 0x7f6df00b5c90 2023-07-06T20:14:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6df67ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df00cd490 con 0x7f6df015fca0 2023-07-06T20:14:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6df67ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df00cd770 con 0x7f6df00b6110 2023-07-06T20:14:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6de7fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6df00b5c90 0x7f6df00d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45356/0 (socket says 172.21.15.27:45356) 2023-07-06T20:14:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6de7fff700 1 -- 172.21.15.27:0/3803085357 learned_addr learned my addr 172.21.15.27:0/3803085357 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.138+0000 7f6df4d9c700 1 --2- 172.21.15.27:0/3803085357 >> v2:172.21.15.27:3300/0 conn(0x7f6df00b6110 0x7f6df00cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6df4d9c700 1 -- 172.21.15.27:0/3803085357 >> v1:172.21.15.153:6789/0 conn(0x7f6df015fca0 legacy=0x7f6df00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6df4d9c700 1 -- 172.21.15.27:0/3803085357 >> v1:172.21.15.27:6789/0 conn(0x7f6df00b5c90 legacy=0x7f6df00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6df4d9c700 1 -- 172.21.15.27:0/3803085357 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00cda50 con 0x7f6df00b6110 2023-07-06T20:14:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6def7fe700 1 -- 172.21.15.27:0/3803085357 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170684152 0 0) 0x7f6df00cf780 con 0x7f6df00b5c90 2023-07-06T20:14:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6df4d9c700 1 --2- 172.21.15.27:0/3803085357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00b6110 0x7f6df00cfc10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009d90 tx=0x7f6de800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44641f1480d9c62d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:55.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6def7fe700 1 -- 172.21.15.27:0/3803085357 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800e070 con 0x7f6df00b6110 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6def7fe700 1 -- 172.21.15.27:0/3803085357 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800bbd0 con 0x7f6df00b6110 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.139+0000 7f6def7fe700 1 -- 172.21.15.27:0/3803085357 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8012430 con 0x7f6df00b6110 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 -- 172.21.15.27:0/3803085357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00b6110 msgr2=0x7f6df00cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 --2- 172.21.15.27:0/3803085357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00b6110 0x7f6df00cfc10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009d90 tx=0x7f6de800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 -- 172.21.15.27:0/3803085357 shutdown_connections 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 --2- 172.21.15.27:0/3803085357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df00b6110 0x7f6df00cfc10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 -- 172.21.15.27:0/3803085357 >> 172.21.15.27:0/3803085357 conn(0x7f6df006c150 msgr2=0x7f6df006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 -- 172.21.15.27:0/3803085357 shutdown_connections 2023-07-06T20:14:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.140+0000 7f6df67ff700 1 -- 172.21.15.27:0/3803085357 wait complete. 2023-07-06T20:14:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.142+0000 7f6df67ff700 1 Processor -- start 2023-07-06T20:14:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.142+0000 7f6df67ff700 1 -- start start 2023-07-06T20:14:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df67ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b5c90 0x7f6df00c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df67ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b6110 0x7f6df01e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df67ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 0x7f6df01e6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df67ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df0145e20 con 0x7f6df015fca0 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df67ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00c7100 con 0x7f6df00b6110 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df67ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00cc730 con 0x7f6df00b5c90 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df4d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 0x7f6df01e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df4d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 0x7f6df01e6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59328/0 (socket says 172.21.15.27:59328) 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.143+0000 7f6df4d9c700 1 -- 172.21.15.27:0/2051157718 learned_addr learned my addr 172.21.15.27:0/2051157718 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6de7fff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b6110 0x7f6df01e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6deffff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b5c90 0x7f6df00c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df4d9c700 1 -- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b5c90 msgr2=0x7f6df00c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df4d9c700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b5c90 0x7f6df00c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df4d9c700 1 -- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b6110 msgr2=0x7f6df01e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df4d9c700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b6110 0x7f6df01e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df4d9c700 1 -- 172.21.15.27:0/2051157718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8009a70 con 0x7f6df015fca0 2023-07-06T20:14:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6de7fff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b6110 0x7f6df01e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6deffff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b5c90 0x7f6df00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df4d9c700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 0x7f6df01e6420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6df006d0f0 tx=0x7f6de8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de801b810 con 0x7f6df015fca0 2023-07-06T20:14:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8020070 con 0x7f6df015fca0 2023-07-06T20:14:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de801bbb0 con 0x7f6df015fca0 2023-07-06T20:14:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.144+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df0146070 con 0x7f6df015fca0 2023-07-06T20:14:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.145+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df0146630 con 0x7f6df015fca0 2023-07-06T20:14:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.146+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df00008d0 con 0x7f6df015fca0 2023-07-06T20:14:55.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.146+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6de801bd10 con 0x7f6df015fca0 2023-07-06T20:14:55.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.147+0000 7f6ded7fa700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6dd006ace0 0x7f6dd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.148+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6de8022070 con 0x7f6df015fca0 2023-07-06T20:14:55.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.148+0000 7f6deffff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6dd006ace0 0x7f6dd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.149+0000 7f6deffff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6dd006ace0 0x7f6dd006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6de0000f60 tx=0x7f6de0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:55.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.152+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6de805a180 con 0x7f6df015fca0 2023-07-06T20:14:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.361+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6df006a850 con 0x7f6df015fca0 2023-07-06T20:14:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.362+0000 7f6ded7fa700 1 -- 172.21.15.27:0/2051157718 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6de8059d10 con 0x7f6df015fca0 2023-07-06T20:14:55.364 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:55.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6dd006ace0 msgr2=0x7f6dd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6dd006ace0 0x7f6dd006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6de0000f60 tx=0x7f6de0009040 comp rx=0 tx=0).stop 2023-07-06T20:14:55.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 msgr2=0x7f6df01e6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 0x7f6df01e6420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6df006d0f0 tx=0x7f6de8004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 shutdown_connections 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6dd006ace0 0x7f6dd006d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6df015fca0 0x7f6df01e6420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6df00b5c90 0x7f6df00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 --2- 172.21.15.27:0/2051157718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6df00b6110 0x7f6df01e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 >> 172.21.15.27:0/2051157718 conn(0x7f6df006c150 msgr2=0x7f6df00cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:55.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 shutdown_connections 2023-07-06T20:14:55.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.364+0000 7f6df67ff700 1 -- 172.21.15.27:0/2051157718 wait complete. 2023-07-06T20:14:55.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:14:55.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.587+0000 7fe3f37ff700 1 Processor -- start 2023-07-06T20:14:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.588+0000 7fe3f37ff700 1 -- start start 2023-07-06T20:14:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f37ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3ec15fca0 0x7fe3ec0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f37ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f37ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ec0cf780 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f37ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ec0cd490 con 0x7fe3ec15fca0 2023-07-06T20:14:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f37ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0cd770 con 0x7fe3ec0b6110 2023-07-06T20:14:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f159b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3ec15fca0 0x7fe3ec0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f0d9a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe3ec0b5c90 0x7fe3ec0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45372/0 (socket says 172.21.15.27:45372) 2023-07-06T20:14:55.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f0d9a700 1 -- 172.21.15.27:0/1683914083 learned_addr learned my addr 172.21.15.27:0/1683914083 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:55.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.589+0000 7fe3f1d9c700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3f159b700 1 -- 172.21.15.27:0/1683914083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 msgr2=0x7fe3ec0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:55.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3f159b700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3f159b700 1 -- 172.21.15.27:0/1683914083 >> v1:172.21.15.27:6789/0 conn(0x7fe3ec0b5c90 legacy=0x7fe3ec0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3f159b700 1 -- 172.21.15.27:0/1683914083 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0cda50 con 0x7fe3ec15fca0 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3f1d9c700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3f159b700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec15fca0 0x7fe3ec0d1420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009840 tx=0x7fe3e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4980f0a43aa2318 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3e3fff700 1 -- 172.21.15.27:0/1683914083 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8011830 con 0x7fe3ec15fca0 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3e3fff700 1 -- 172.21.15.27:0/1683914083 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e8011990 con 0x7fe3ec15fca0 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.590+0000 7fe3e3fff700 1 -- 172.21.15.27:0/1683914083 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8011d20 con 0x7fe3ec15fca0 2023-07-06T20:14:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 -- 172.21.15.27:0/1683914083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec15fca0 msgr2=0x7fe3ec0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec15fca0 0x7fe3ec0d1420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009840 tx=0x7fe3e8009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:55.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 -- 172.21.15.27:0/1683914083 shutdown_connections 2023-07-06T20:14:55.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/1683914083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec15fca0 0x7fe3ec0d1420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 -- 172.21.15.27:0/1683914083 >> 172.21.15.27:0/1683914083 conn(0x7fe3ec06c150 msgr2=0x7fe3ec06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:55.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 -- 172.21.15.27:0/1683914083 shutdown_connections 2023-07-06T20:14:55.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.591+0000 7fe3f37ff700 1 -- 172.21.15.27:0/1683914083 wait complete. 2023-07-06T20:14:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.593+0000 7fe3f37ff700 1 Processor -- start 2023-07-06T20:14:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.593+0000 7fe3f37ff700 1 -- start start 2023-07-06T20:14:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f37ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 0x7fe3ec1e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f37ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f37ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ec15fca0 0x7fe3ec1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f37ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0c6e60 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f37ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0c7140 con 0x7fe3ec15fca0 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f37ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0c7420 con 0x7fe3ec0b6110 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 0x7fe3ec1e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 0x7fe3ec1e68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59348/0 (socket says 172.21.15.27:59348) 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 -- 172.21.15.27:0/2076945166 learned_addr learned my addr 172.21.15.27:0/2076945166 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 -- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 msgr2=0x7fe3ec0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 -- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ec15fca0 msgr2=0x7fe3ec1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ec15fca0 0x7fe3ec1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.594+0000 7fe3f159b700 1 -- 172.21.15.27:0/2076945166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80094a0 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.595+0000 7fe3f159b700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 0x7fe3ec1e68f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000c00 tx=0x7fe3e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.595+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80038a0 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.595+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e801e070 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.595+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8003c60 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.595+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ec0c7700 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.595+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ec1e6e30 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.596+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec0c7c20 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.597+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3e802d450 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.598+0000 7fe3e1ffb700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe3d006ace0 0x7fe3d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.599+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe3e808a6b0 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.599+0000 7fe3f0d9a700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe3d006ace0 0x7fe3d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.600+0000 7fe3f0d9a700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe3d006ace0 0x7fe3d006d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0ccef0 tx=0x7fe3dc007ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:55.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.603+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3ec0c7c20 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.823+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3ec0c7c20 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.823+0000 7fe3e1ffb700 1 -- 172.21.15.27:0/2076945166 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe3ec0c7c20 con 0x7fe3ec0b5c90 2023-07-06T20:14:55.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.825+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe3d006ace0 msgr2=0x7fe3d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.825+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe3d006ace0 0x7fe3d006d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0ccef0 tx=0x7fe3dc007ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.825+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 msgr2=0x7fe3ec1e68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:55.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.825+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 0x7fe3ec1e68f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000c00 tx=0x7fe3e8004750 comp rx=0 tx=0).stop 2023-07-06T20:14:55.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 shutdown_connections 2023-07-06T20:14:55.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe3d006ace0 0x7fe3d006d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ec15fca0 0x7fe3ec1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3ec0b5c90 0x7fe3ec1e68f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 --2- 172.21.15.27:0/2076945166 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ec0b6110 0x7fe3ec0b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:55.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 >> 172.21.15.27:0/2076945166 conn(0x7fe3ec06c150 msgr2=0x7fe3ec0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:55.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 shutdown_connections 2023-07-06T20:14:55.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:55.826+0000 7fe3f37ff700 1 -- 172.21.15.27:0/2076945166 wait complete. 2023-07-06T20:14:55.830 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-07-06T20:14:55.845 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-07-06T20:14:55.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:55.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-07-06T20:14:55.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:14:55.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:55.930 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T20:14:55.977 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:55.977 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:56.029 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:56.029 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:56.081 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:14:56.081 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:14:56.127 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.330+0000 7f21d3dff700 1 Processor -- start 2023-07-06T20:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.330+0000 7f21d3dff700 1 -- start start 2023-07-06T20:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.331+0000 7f21d3dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.331+0000 7f21d3dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc0cf780 con 0x7f21cc0b60e0 2023-07-06T20:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.331+0000 7f21d3dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc0cd490 con 0x7f21cc15fca0 2023-07-06T20:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.331+0000 7f21d3dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0cd770 con 0x7f21cc0b5c90 2023-07-06T20:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.331+0000 7f21d239c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f21cc0b60e0 0x7f21cc0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45384/0 (socket says 172.21.15.27:45384) 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.331+0000 7f21d239c700 1 -- 172.21.15.27:0/2321634665 learned_addr learned my addr 172.21.15.27:0/2321634665 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d139a700 1 --2- 172.21.15.27:0/2321634665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164722502 0 0) 0x7f21cc0cf780 con 0x7f21cc0b60e0 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b80035b0 con 0x7f21cc0b60e0 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980073908 0 0) 0x7f21cc0cd490 con 0x7f21cc15fca0 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21cc0cf780 con 0x7f21cc15fca0 2023-07-06T20:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d139a700 1 -- 172.21.15.27:0/2321634665 >> v1:172.21.15.153:6789/0 conn(0x7f21cc15fca0 legacy=0x7f21cc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187584680 0 0) 0x7f21b80035b0 con 0x7f21cc0b60e0 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d139a700 1 -- 172.21.15.27:0/2321634665 >> v1:172.21.15.27:6789/0 conn(0x7f21cc0b60e0 legacy=0x7f21cc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.332+0000 7f21d139a700 1 -- 172.21.15.27:0/2321634665 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc0cda50 con 0x7f21cc0b5c90 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.333+0000 7f21d139a700 1 --2- 172.21.15.27:0/2321634665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0d2d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009870 tx=0x7f21bc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd80a34cdadcb3d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.333+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40019f0 con 0x7f21cc0b5c90 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.333+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc004030 con 0x7f21cc0b5c90 2023-07-06T20:14:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.333+0000 7f21d0b99700 1 -- 172.21.15.27:0/2321634665 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0043c0 con 0x7f21cc0b5c90 2023-07-06T20:14:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.333+0000 7f21d3dff700 1 -- 172.21.15.27:0/2321634665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 msgr2=0x7f21cc0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.333+0000 7f21d3dff700 1 --2- 172.21.15.27:0/2321634665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0d2d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009870 tx=0x7f21bc009b80 comp rx=0 tx=0).stop 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.334+0000 7f21d3dff700 1 -- 172.21.15.27:0/2321634665 shutdown_connections 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.334+0000 7f21d3dff700 1 --2- 172.21.15.27:0/2321634665 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0d2d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.334+0000 7f21d3dff700 1 -- 172.21.15.27:0/2321634665 >> 172.21.15.27:0/2321634665 conn(0x7f21cc06c150 msgr2=0x7f21cc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.334+0000 7f21d3dff700 1 -- 172.21.15.27:0/2321634665 shutdown_connections 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.334+0000 7f21d3dff700 1 -- 172.21.15.27:0/2321634665 wait complete. 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.335+0000 7f21d3dff700 1 Processor -- start 2023-07-06T20:14:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.336+0000 7f21d3dff700 1 -- start start 2023-07-06T20:14:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d3dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d3dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 0x7f21cc1e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d3dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21cc15fca0 0x7f21cc1e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d3dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1e8d50 con 0x7f21cc15fca0 2023-07-06T20:14:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d3dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0c9110 con 0x7f21cc0b60e0 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d3dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1e84b0 con 0x7f21cc0b5c90 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d239c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21cc15fca0 0x7f21cc1e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 0x7f21cc1e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d1b9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 0x7f21cc1e7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57810/0 (socket says 172.21.15.27:57810) 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 -- 172.21.15.27:0/1669122599 learned_addr learned my addr 172.21.15.27:0/1669122599 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 -- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 msgr2=0x7f21cc0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 -- 172.21.15.27:0/1669122599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21cc15fca0 msgr2=0x7f21cc1e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21cc15fca0 0x7f21cc1e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.337+0000 7f21d139a700 1 -- 172.21.15.27:0/1669122599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0094a0 con 0x7f21cc0b60e0 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.338+0000 7f21d139a700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 0x7f21cc1e7940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f21bc000c00 tx=0x7f21bc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.338+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc003e60 con 0x7f21cc0b60e0 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.338+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc01f070 con 0x7f21cc0b60e0 2023-07-06T20:14:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.338+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc011850 con 0x7f21cc0b60e0 2023-07-06T20:14:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.338+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc1e8790 con 0x7f21cc0b60e0 2023-07-06T20:14:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.338+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc143ba0 con 0x7f21cc0b60e0 2023-07-06T20:14:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.339+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc143a00 con 0x7f21cc0b60e0 2023-07-06T20:14:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.341+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21bc0119b0 con 0x7f21cc0b60e0 2023-07-06T20:14:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.341+0000 7f21c27fc700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b006ace0 0x7f21b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.342+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f21bc021070 con 0x7f21cc0b60e0 2023-07-06T20:14:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.342+0000 7f21d1b9b700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b006ace0 0x7f21b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.342+0000 7f21d1b9b700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b006ace0 0x7f21b006d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21c80011e0 tx=0x7f21c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:56.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.345+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21bc055e50 con 0x7f21cc0b60e0 2023-07-06T20:14:56.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.553+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21cc143a00 con 0x7f21cc0b60e0 2023-07-06T20:14:56.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.556+0000 7f21c27fc700 1 -- 172.21.15.27:0/1669122599 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f21cc143a00 con 0x7f21cc0b60e0 2023-07-06T20:14:56.557 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:56.558 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:56.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.557+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b006ace0 msgr2=0x7f21b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.557+0000 7f21d3dff700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b006ace0 0x7f21b006d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21c80011e0 tx=0x7f21c8009040 comp rx=0 tx=0).stop 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.557+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 msgr2=0x7f21cc1e7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.557+0000 7f21d3dff700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 0x7f21cc1e7940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f21bc000c00 tx=0x7f21bc004750 comp rx=0 tx=0).stop 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 shutdown_connections 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f21b006ace0 0x7f21b006d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21cc15fca0 0x7f21cc1e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f21cc0b5c90 0x7f21cc0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 --2- 172.21.15.27:0/1669122599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21cc0b60e0 0x7f21cc1e7940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 >> 172.21.15.27:0/1669122599 conn(0x7f21cc06c150 msgr2=0x7f21cc0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 shutdown_connections 2023-07-06T20:14:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.558+0000 7f21d3dff700 1 -- 172.21.15.27:0/1669122599 wait complete. 2023-07-06T20:14:56.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:56.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:14:56.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.783+0000 7fb6169ff700 1 Processor -- start 2023-07-06T20:14:56.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.783+0000 7fb6169ff700 1 -- start start 2023-07-06T20:14:56.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb6169ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 0x7fb6100d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb6169ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb6100b60e0 0x7fb6100d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb6169ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6100cf780 con 0x7fb61015fca0 2023-07-06T20:14:56.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb6169ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100cd490 con 0x7fb6100b60e0 2023-07-06T20:14:56.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb6169ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100cd770 con 0x7fb6100b5c90 2023-07-06T20:14:56.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb614f9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb6100b60e0 0x7fb6100d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb60f7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 0x7fb6100d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb60ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb61015fca0 0x7fb6100d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:46914/0 (socket says 172.21.15.27:46914) 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb60ffff700 1 -- 172.21.15.27:0/2498714240 learned_addr learned my addr 172.21.15.27:0/2498714240 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb60f7fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 0x7fb6100d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:42496/0 (socket says 172.21.15.27:42496) 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.784+0000 7fb614f9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb6100b60e0 0x7fb6100d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57824/0 (socket says 172.21.15.27:57824) 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb60effd700 1 -- 172.21.15.27:0/2498714240 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616974988 0 0) 0x7fb6100cf780 con 0x7fb61015fca0 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb60effd700 1 -- 172.21.15.27:0/2498714240 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5f80035b0 con 0x7fb61015fca0 2023-07-06T20:14:56.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb614f9c700 1 -- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 msgr2=0x7fb6100d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb614f9c700 1 --2- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 0x7fb6100d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb614f9c700 1 -- 172.21.15.27:0/2498714240 >> v1:172.21.15.153:6789/0 conn(0x7fb61015fca0 legacy=0x7fb6100d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb614f9c700 1 -- 172.21.15.27:0/2498714240 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6100cda50 con 0x7fb6100b60e0 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb60f7fe700 1 --2- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 0x7fb6100d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.785+0000 7fb614f9c700 1 --2- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100d3280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb608009d90 tx=0x7fb60800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a382a603245f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb60effd700 1 -- 172.21.15.27:0/2498714240 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800e070 con 0x7fb6100b60e0 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb60effd700 1 -- 172.21.15.27:0/2498714240 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60800bb20 con 0x7fb6100b60e0 2023-07-06T20:14:56.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb60effd700 1 -- 172.21.15.27:0/2498714240 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608012430 con 0x7fb6100b60e0 2023-07-06T20:14:56.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 -- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 msgr2=0x7fb6100d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 --2- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100d3280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb608009d90 tx=0x7fb60800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 -- 172.21.15.27:0/2498714240 shutdown_connections 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 --2- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100d3280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 --2- 172.21.15.27:0/2498714240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6100b5c90 0x7fb6100d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 -- 172.21.15.27:0/2498714240 >> 172.21.15.27:0/2498714240 conn(0x7fb61006c150 msgr2=0x7fb61006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 -- 172.21.15.27:0/2498714240 shutdown_connections 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.786+0000 7fb6169ff700 1 -- 172.21.15.27:0/2498714240 wait complete. 2023-07-06T20:14:56.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.788+0000 7fb6169ff700 1 Processor -- start 2023-07-06T20:14:56.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.788+0000 7fb6169ff700 1 -- start start 2023-07-06T20:14:56.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb6169ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 0x7fb61015f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb6169ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb6169ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61015fca0 0x7fb6100b8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb6169ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101f2930 con 0x7fb6100b5c90 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb6169ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101f2650 con 0x7fb6100b60e0 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb6169ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b76a0 con 0x7fb61015fca0 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.789+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 0x7fb61015f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 0x7fb61015f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59364/0 (socket says 172.21.15.27:59364) 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 -- 172.21.15.27:0/1739554260 learned_addr learned my addr 172.21.15.27:0/1739554260 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60f7fe700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb614f9c700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61015fca0 0x7fb6100b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 -- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61015fca0 msgr2=0x7fb6100b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61015fca0 0x7fb6100b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 -- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 msgr2=0x7fb6100b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 -- 172.21.15.27:0/1739554260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040094a0 con 0x7fb6100b5c90 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60ffff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 0x7fb61015f110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb600001eb0 tx=0x7fb60000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600005d70 con 0x7fb6100b5c90 2023-07-06T20:14:56.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6000070d0 con 0x7fb6100b5c90 2023-07-06T20:14:56.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600007460 con 0x7fb6100b5c90 2023-07-06T20:14:56.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.790+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb608009a70 con 0x7fb6100b5c90 2023-07-06T20:14:56.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.791+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6100b7c60 con 0x7fb6100b5c90 2023-07-06T20:14:56.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.792+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6100008d0 con 0x7fb6100b5c90 2023-07-06T20:14:56.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.793+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6000075c0 con 0x7fb6100b5c90 2023-07-06T20:14:56.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.794+0000 7fb60cff9700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb5fc06ace0 0x7fb5fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:56.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.794+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb600088a90 con 0x7fb6100b5c90 2023-07-06T20:14:56.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.794+0000 7fb60f7fe700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb5fc06ace0 0x7fb5fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:56.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.795+0000 7fb60f7fe700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb5fc06ace0 0x7fb5fc06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb610159c20 tx=0x7fb604008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:56.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:56.799+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb600053af0 con 0x7fb6100b5c90 2023-07-06T20:14:57.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.006+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6100b45e0 con 0x7fb6100b5c90 2023-07-06T20:14:57.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.006+0000 7fb60cff9700 1 -- 172.21.15.27:0/1739554260 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb600057110 con 0x7fb6100b5c90 2023-07-06T20:14:57.009 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:14:57.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb5fc06ace0 msgr2=0x7fb5fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb5fc06ace0 0x7fb5fc06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb610159c20 tx=0x7fb604008040 comp rx=0 tx=0).stop 2023-07-06T20:14:57.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 msgr2=0x7fb61015f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 0x7fb61015f110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb600001eb0 tx=0x7fb60000c860 comp rx=0 tx=0).stop 2023-07-06T20:14:57.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 shutdown_connections 2023-07-06T20:14:57.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb5fc06ace0 0x7fb5fc06d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61015fca0 0x7fb6100b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6100b5c90 0x7fb61015f110 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 --2- 172.21.15.27:0/1739554260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6100b60e0 0x7fb6100b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 >> 172.21.15.27:0/1739554260 conn(0x7fb61006c150 msgr2=0x7fb6100d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:57.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 shutdown_connections 2023-07-06T20:14:57.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.009+0000 7fb6169ff700 1 -- 172.21.15.27:0/1739554260 wait complete. 2023-07-06T20:14:57.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:14:57.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.236+0000 7fb81afff700 1 Processor -- start 2023-07-06T20:14:57.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.236+0000 7fb81afff700 1 -- start start 2023-07-06T20:14:57.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.237+0000 7fb81afff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb81415fca0 0x7fb8140d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.237+0000 7fb81afff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8140b5c90 0x7fb8140d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb81afff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8140cf780 con 0x7fb8140b60e0 2023-07-06T20:14:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb81afff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8140cd490 con 0x7fb81415fca0 2023-07-06T20:14:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb81afff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8140cd770 con 0x7fb8140b5c90 2023-07-06T20:14:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb81959c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb8140b60e0 0x7fb8140cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45392/0 (socket says 172.21.15.27:45392) 2023-07-06T20:14:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb81959c700 1 -- 172.21.15.27:0/1457080407 learned_addr learned my addr 172.21.15.27:0/1457080407 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb813fff700 1 --2- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8140b5c90 0x7fb8140d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb818d9b700 1 --2- 172.21.15.27:0/1457080407 >> v2:172.21.15.153:3300/0 conn(0x7fb81415fca0 0x7fb8140d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb8137fe700 1 -- 172.21.15.27:0/1457080407 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850569195 0 0) 0x7fb8140cf780 con 0x7fb8140b60e0 2023-07-06T20:14:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.238+0000 7fb8137fe700 1 -- 172.21.15.27:0/1457080407 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8040035b0 con 0x7fb8140b60e0 2023-07-06T20:14:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb818d9b700 1 -- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8140b5c90 msgr2=0x7fb8140d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb818d9b700 1 --2- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8140b5c90 0x7fb8140d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb818d9b700 1 -- 172.21.15.27:0/1457080407 >> v1:172.21.15.27:6789/0 conn(0x7fb8140b60e0 legacy=0x7fb8140cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb8137fe700 1 -- 172.21.15.27:0/1457080407 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2697514197 0 0) 0x7fb8040035b0 con 0x7fb8140b60e0 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb818d9b700 1 -- 172.21.15.27:0/1457080407 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8140cda50 con 0x7fb81415fca0 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb818d9b700 1 --2- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81415fca0 0x7fb8140d1420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb808009840 tx=0x7fb808009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708800811f93e3f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb8137fe700 1 -- 172.21.15.27:0/1457080407 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0019f0 con 0x7fb81415fca0 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb8137fe700 1 -- 172.21.15.27:0/1457080407 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808004030 con 0x7fb81415fca0 2023-07-06T20:14:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.239+0000 7fb8137fe700 1 -- 172.21.15.27:0/1457080407 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8080043c0 con 0x7fb81415fca0 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 -- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81415fca0 msgr2=0x7fb8140d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 --2- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81415fca0 0x7fb8140d1420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb808009840 tx=0x7fb808009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 -- 172.21.15.27:0/1457080407 shutdown_connections 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 --2- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81415fca0 0x7fb8140d1420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 --2- 172.21.15.27:0/1457080407 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8140b5c90 0x7fb8140d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 -- 172.21.15.27:0/1457080407 >> 172.21.15.27:0/1457080407 conn(0x7fb81406c150 msgr2=0x7fb81406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 -- 172.21.15.27:0/1457080407 shutdown_connections 2023-07-06T20:14:57.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.240+0000 7fb81afff700 1 -- 172.21.15.27:0/1457080407 wait complete. 2023-07-06T20:14:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.242+0000 7fb81afff700 1 Processor -- start 2023-07-06T20:14:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.242+0000 7fb81afff700 1 -- start start 2023-07-06T20:14:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb81afff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 0x7fb8140c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb81afff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8140b60e0 0x7fb8140b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb81afff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb81415fca0 0x7fb81415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb81afff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8140cb250 con 0x7fb8140b5c90 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb81afff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81415bd10 con 0x7fb8140b60e0 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb81afff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8140ca920 con 0x7fb81415fca0 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb818d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 0x7fb8140c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb818d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 0x7fb8140c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59372/0 (socket says 172.21.15.27:59372) 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.243+0000 7fb818d9b700 1 -- 172.21.15.27:0/227779066 learned_addr learned my addr 172.21.15.27:0/227779066 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb813fff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8140b60e0 0x7fb8140b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb81959c700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb81415fca0 0x7fb81415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb818d9b700 1 -- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb81415fca0 msgr2=0x7fb81415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb818d9b700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb81415fca0 0x7fb81415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb818d9b700 1 -- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8140b60e0 msgr2=0x7fb8140b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb818d9b700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8140b60e0 0x7fb8140b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb818d9b700 1 -- 172.21.15.27:0/227779066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080094a0 con 0x7fb8140b5c90 2023-07-06T20:14:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb818d9b700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 0x7fb8140c8cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb80800a010 tx=0x7fb808004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808003c00 con 0x7fb8140b5c90 2023-07-06T20:14:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80801f070 con 0x7fb8140b5c90 2023-07-06T20:14:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8140cac00 con 0x7fb8140b5c90 2023-07-06T20:14:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.244+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808011630 con 0x7fb8140b5c90 2023-07-06T20:14:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.245+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8140b7ab0 con 0x7fb8140b5c90 2023-07-06T20:14:57.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.246+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8140008d0 con 0x7fb8140b5c90 2023-07-06T20:14:57.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.246+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb808021030 con 0x7fb8140b5c90 2023-07-06T20:14:57.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.248+0000 7fb8117fa700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb7f806ace0 0x7fb7f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.248+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb80808a7b0 con 0x7fb8140b5c90 2023-07-06T20:14:57.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.248+0000 7fb813fff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb7f806ace0 0x7fb7f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.249+0000 7fb813fff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb7f806ace0 0x7fb7f806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb81414cfd0 tx=0x7fb800008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.252+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb808055810 con 0x7fb8140b5c90 2023-07-06T20:14:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.459+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb8140732b0 con 0x7fb8140b5c90 2023-07-06T20:14:57.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.460+0000 7fb8117fa700 1 -- 172.21.15.27:0/227779066 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb808058e30 con 0x7fb8140b5c90 2023-07-06T20:14:57.462 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:14:57.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb7f806ace0 msgr2=0x7fb7f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb7f806ace0 0x7fb7f806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb81414cfd0 tx=0x7fb800008040 comp rx=0 tx=0).stop 2023-07-06T20:14:57.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 msgr2=0x7fb8140c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 0x7fb8140c8cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb80800a010 tx=0x7fb808004750 comp rx=0 tx=0).stop 2023-07-06T20:14:57.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 shutdown_connections 2023-07-06T20:14:57.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb7f806ace0 0x7fb7f806d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb81415fca0 0x7fb81415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8140b5c90 0x7fb8140c8cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 --2- 172.21.15.27:0/227779066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8140b60e0 0x7fb8140b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 >> 172.21.15.27:0/227779066 conn(0x7fb81406c150 msgr2=0x7fb8140b8780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 shutdown_connections 2023-07-06T20:14:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.462+0000 7fb81afff700 1 -- 172.21.15.27:0/227779066 wait complete. 2023-07-06T20:14:57.482 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:14:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.689+0000 7f39f2bff700 1 Processor -- start 2023-07-06T20:14:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.689+0000 7f39f2bff700 1 -- start start 2023-07-06T20:14:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f2bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39ec15fca0 0x7f39ec0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f2bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b6110 0x7f39ec0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f2bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ec0cf780 con 0x7f39ec0b5c90 2023-07-06T20:14:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f2bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ec0cd490 con 0x7f39ec15fca0 2023-07-06T20:14:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f2bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec0cd770 con 0x7f39ec0b6110 2023-07-06T20:14:57.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f099b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39ec15fca0 0x7f39ec0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f099b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39ec15fca0 0x7f39ec0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59382/0 (socket says 172.21.15.27:59382) 2023-07-06T20:14:57.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.690+0000 7f39f099b700 1 -- 172.21.15.27:0/3084756957 learned_addr learned my addr 172.21.15.27:0/3084756957 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:57.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39f119c700 1 --2- 172.21.15.27:0/3084756957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b6110 0x7f39ec0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39f099b700 1 -- 172.21.15.27:0/3084756957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b6110 msgr2=0x7f39ec0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:57.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39f099b700 1 --2- 172.21.15.27:0/3084756957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b6110 0x7f39ec0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39f099b700 1 -- 172.21.15.27:0/3084756957 >> v1:172.21.15.153:6789/0 conn(0x7f39ec0b5c90 legacy=0x7f39ec0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39f099b700 1 -- 172.21.15.27:0/3084756957 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec0cda50 con 0x7f39ec15fca0 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39f099b700 1 --2- 172.21.15.27:0/3084756957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec15fca0 0x7f39ec0d1420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098f0 tx=0x7f39e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d526630e458ddd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39eb7fe700 1 -- 172.21.15.27:0/3084756957 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0011830 con 0x7f39ec15fca0 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39eb7fe700 1 -- 172.21.15.27:0/3084756957 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0011990 con 0x7f39ec15fca0 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.691+0000 7f39eb7fe700 1 -- 172.21.15.27:0/3084756957 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0011d20 con 0x7f39ec15fca0 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 -- 172.21.15.27:0/3084756957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec15fca0 msgr2=0x7f39ec0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 --2- 172.21.15.27:0/3084756957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec15fca0 0x7f39ec0d1420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098f0 tx=0x7f39e0009c00 comp rx=0 tx=0).stop 2023-07-06T20:14:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 -- 172.21.15.27:0/3084756957 shutdown_connections 2023-07-06T20:14:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 --2- 172.21.15.27:0/3084756957 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b6110 0x7f39ec0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 --2- 172.21.15.27:0/3084756957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec15fca0 0x7f39ec0d1420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 -- 172.21.15.27:0/3084756957 >> 172.21.15.27:0/3084756957 conn(0x7f39ec06c150 msgr2=0x7f39ec06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 -- 172.21.15.27:0/3084756957 shutdown_connections 2023-07-06T20:14:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.692+0000 7f39f2bff700 1 -- 172.21.15.27:0/3084756957 wait complete. 2023-07-06T20:14:57.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.694+0000 7f39f2bff700 1 Processor -- start 2023-07-06T20:14:57.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.694+0000 7f39f2bff700 1 -- start start 2023-07-06T20:14:57.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f2bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b5c90 0x7f39ec148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f2bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 0x7f39ec15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f2bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ec15fca0 0x7f39ec14ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f2bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec15da20 con 0x7f39ec0b6110 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f2bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec14b7d0 con 0x7f39ec15fca0 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f2bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec14bab0 con 0x7f39ec0b5c90 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 0x7f39ec15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 0x7f39ec15deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59392/0 (socket says 172.21.15.27:59392) 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39ebfff700 1 -- 172.21.15.27:0/2261891762 learned_addr learned my addr 172.21.15.27:0/2261891762 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.695+0000 7f39f099b700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b5c90 0x7f39ec148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39f119c700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ec15fca0 0x7f39ec14ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39ebfff700 1 -- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b5c90 msgr2=0x7f39ec148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39ebfff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b5c90 0x7f39ec148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39ebfff700 1 -- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ec15fca0 msgr2=0x7f39ec14ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39ebfff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ec15fca0 0x7f39ec14ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39ebfff700 1 -- 172.21.15.27:0/2261891762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0009550 con 0x7f39ec0b6110 2023-07-06T20:14:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39f099b700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b5c90 0x7f39ec148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39f119c700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ec15fca0 0x7f39ec14ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39ebfff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 0x7f39ec15deb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39d800c4e0 tx=0x7f39d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80052d0 con 0x7f39ec0b6110 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8005430 con 0x7f39ec0b6110 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80038c0 con 0x7f39ec0b6110 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.696+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec15d6b0 con 0x7f39ec0b6110 2023-07-06T20:14:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.697+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec15b9d0 con 0x7f39ec0b6110 2023-07-06T20:14:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.697+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec15db90 con 0x7f39ec0b6110 2023-07-06T20:14:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.698+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39d8003a20 con 0x7f39ec0b6110 2023-07-06T20:14:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.700+0000 7f39e97fa700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f39d006ace0 0x7f39d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:57.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.700+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f39d8088a80 con 0x7f39ec0b6110 2023-07-06T20:14:57.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.700+0000 7f39f099b700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f39d006ace0 0x7f39d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:57.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.701+0000 7f39f099b700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f39d006ace0 0x7f39d006d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39e0014040 tx=0x7f39e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:57.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.703+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f39d8053a50 con 0x7f39ec0b6110 2023-07-06T20:14:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.911+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39ec0008d0 con 0x7f39ec0b6110 2023-07-06T20:14:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.911+0000 7f39e97fa700 1 -- 172.21.15.27:0/2261891762 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39d8057070 con 0x7f39ec0b6110 2023-07-06T20:14:57.913 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:14:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f39d006ace0 msgr2=0x7f39d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f39d006ace0 0x7f39d006d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39e0014040 tx=0x7f39e0008030 comp rx=0 tx=0).stop 2023-07-06T20:14:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 msgr2=0x7f39ec15deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 0x7f39ec15deb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39d800c4e0 tx=0x7f39d800c8a0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 shutdown_connections 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f39d006ace0 0x7f39d006d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ec15fca0 0x7f39ec14ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f39ec0b5c90 0x7f39ec148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 --2- 172.21.15.27:0/2261891762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39ec0b6110 0x7f39ec15deb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 >> 172.21.15.27:0/2261891762 conn(0x7f39ec06c150 msgr2=0x7f39ec0b7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 shutdown_connections 2023-07-06T20:14:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:57.914+0000 7f39f2bff700 1 -- 172.21.15.27:0/2261891762 wait complete. 2023-07-06T20:14:57.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:14:58.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.140+0000 7f6a82fff700 1 Processor -- start 2023-07-06T20:14:58.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.140+0000 7f6a82fff700 1 -- start start 2023-07-06T20:14:58.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.141+0000 7f6a82fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c15fca0 0x7f6a7c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.141+0000 7f6a82fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b6100 0x7f6a7c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.141+0000 7f6a82fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0cf780 con 0x7f6a7c0b5c90 2023-07-06T20:14:58.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.141+0000 7f6a82fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0cd490 con 0x7f6a7c0b6100 2023-07-06T20:14:58.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.141+0000 7f6a82fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0cd770 con 0x7f6a7c15fca0 2023-07-06T20:14:58.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a7bfff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6a7c0b5c90 0x7f6a7c0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45408/0 (socket says 172.21.15.27:45408) 2023-07-06T20:14:58.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a7bfff700 1 -- 172.21.15.27:0/3010817701 learned_addr learned my addr 172.21.15.27:0/3010817701 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:58.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a80d9b700 1 --2- 172.21.15.27:0/3010817701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c15fca0 0x7f6a7c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a8159c700 1 --2- 172.21.15.27:0/3010817701 >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b6100 0x7f6a7c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a7b7fe700 1 -- 172.21.15.27:0/3010817701 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165357316 0 0) 0x7f6a7c0cf780 con 0x7f6a7c0b5c90 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a7b7fe700 1 -- 172.21.15.27:0/3010817701 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a6c0035b0 con 0x7f6a7c0b5c90 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a80d9b700 1 -- 172.21.15.27:0/3010817701 >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b6100 msgr2=0x7f6a7c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a80d9b700 1 --2- 172.21.15.27:0/3010817701 >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b6100 0x7f6a7c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a80d9b700 1 -- 172.21.15.27:0/3010817701 >> v1:172.21.15.27:6789/0 conn(0x7f6a7c0b5c90 legacy=0x7f6a7c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.142+0000 7f6a80d9b700 1 -- 172.21.15.27:0/3010817701 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0cda50 con 0x7f6a7c15fca0 2023-07-06T20:14:58.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.143+0000 7f6a80d9b700 1 --2- 172.21.15.27:0/3010817701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c15fca0 0x7f6a7c0d1420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009840 tx=0x7f6a70009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b757f55903ce75 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.143+0000 7f6a7b7fe700 1 -- 172.21.15.27:0/3010817701 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011830 con 0x7f6a7c15fca0 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.143+0000 7f6a7b7fe700 1 -- 172.21.15.27:0/3010817701 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70011990 con 0x7f6a7c15fca0 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.143+0000 7f6a7b7fe700 1 -- 172.21.15.27:0/3010817701 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011d20 con 0x7f6a7c15fca0 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 -- 172.21.15.27:0/3010817701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c15fca0 msgr2=0x7f6a7c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3010817701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c15fca0 0x7f6a7c0d1420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009840 tx=0x7f6a70009b50 comp rx=0 tx=0).stop 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 -- 172.21.15.27:0/3010817701 shutdown_connections 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3010817701 >> v2:172.21.15.153:3300/0 conn(0x7f6a7c0b6100 0x7f6a7c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3010817701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c15fca0 0x7f6a7c0d1420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 -- 172.21.15.27:0/3010817701 >> 172.21.15.27:0/3010817701 conn(0x7f6a7c06c150 msgr2=0x7f6a7c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 -- 172.21.15.27:0/3010817701 shutdown_connections 2023-07-06T20:14:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.144+0000 7f6a82fff700 1 -- 172.21.15.27:0/3010817701 wait complete. 2023-07-06T20:14:58.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.146+0000 7f6a82fff700 1 Processor -- start 2023-07-06T20:14:58.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.146+0000 7f6a82fff700 1 -- start start 2023-07-06T20:14:58.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a82fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b5c90 0x7f6a7c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a82fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c0b6100 0x7f6a7c0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a82fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 0x7f6a7c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a82fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1571f0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a82fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b4330 con 0x7f6a7c0b5c90 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a82fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b2d50 con 0x7f6a7c0b6100 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a8159c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 0x7f6a7c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a8159c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 0x7f6a7c0b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59406/0 (socket says 172.21.15.27:59406) 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a80d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b5c90 0x7f6a7c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a8159c700 1 -- 172.21.15.27:0/3271484258 learned_addr learned my addr 172.21.15.27:0/3271484258 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a7bfff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c0b6100 0x7f6a7c0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a8159c700 1 -- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c0b6100 msgr2=0x7f6a7c0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.147+0000 7f6a8159c700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c0b6100 0x7f6a7c0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a8159c700 1 -- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b5c90 msgr2=0x7f6a7c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a8159c700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b5c90 0x7f6a7c156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a8159c700 1 -- 172.21.15.27:0/3271484258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700094a0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a8159c700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 0x7f6a7c0b4f40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400efc0 tx=0x7f6a7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400cd40 con 0x7f6a7c15fca0 2023-07-06T20:14:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7400f040 con 0x7f6a7c15fca0 2023-07-06T20:14:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c0b3030 con 0x7f6a7c15fca0 2023-07-06T20:14:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740146c0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.148+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0b35f0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.149+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0008d0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.150+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a74014820 con 0x7f6a7c15fca0 2023-07-06T20:14:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.151+0000 7f6a797fa700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6006ace0 0x7f6a6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.151+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6a7408a210 con 0x7f6a7c15fca0 2023-07-06T20:14:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.152+0000 7f6a80d9b700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6006ace0 0x7f6a6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.153+0000 7f6a80d9b700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6006ace0 0x7f6a6006d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a70000c00 tx=0x7f6a70008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:58.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.155+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6a740551e0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.361+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a7c0d2630 con 0x7f6a7c15fca0 2023-07-06T20:14:58.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.362+0000 7f6a797fa700 1 -- 172.21.15.27:0/3271484258 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a74009ca0 con 0x7f6a7c15fca0 2023-07-06T20:14:58.364 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:14:58.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6006ace0 msgr2=0x7f6a6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6006ace0 0x7f6a6006d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a70000c00 tx=0x7f6a70008030 comp rx=0 tx=0).stop 2023-07-06T20:14:58.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 msgr2=0x7f6a7c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 0x7f6a7c0b4f40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400efc0 tx=0x7f6a7400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 shutdown_connections 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6a6006ace0 0x7f6a6006d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7c15fca0 0x7f6a7c0b4f40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a7c0b5c90 0x7f6a7c156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 --2- 172.21.15.27:0/3271484258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a7c0b6100 0x7f6a7c0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 >> 172.21.15.27:0/3271484258 conn(0x7f6a7c06c150 msgr2=0x7f6a7c0b8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 shutdown_connections 2023-07-06T20:14:58.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.364+0000 7f6a82fff700 1 -- 172.21.15.27:0/3271484258 wait complete. 2023-07-06T20:14:58.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:14:58.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.588+0000 7f3497dff700 1 Processor -- start 2023-07-06T20:14:58.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.588+0000 7f3497dff700 1 -- start start 2023-07-06T20:14:58.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.589+0000 7f3497dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f34900cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.589+0000 7f3497dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34900d0220 0x7f34900cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.589+0000 7f3497dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34900d0660 con 0x7f349015c790 2023-07-06T20:14:58.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.589+0000 7f3497dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34900ca500 con 0x7f34900d0220 2023-07-06T20:14:58.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.589+0000 7f3497dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34900cf920 con 0x7f349006a670 2023-07-06T20:14:58.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349639c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34900d0220 0x7f34900cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f3495b9b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f349015c790 0x7f3490069f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45424/0 (socket says 172.21.15.27:45424) 2023-07-06T20:14:58.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f3495b9b700 1 -- 172.21.15.27:0/1782557184 learned_addr learned my addr 172.21.15.27:0/1782557184 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:58.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349539a700 1 --2- 172.21.15.27:0/1782557184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f34900cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349639c700 1 -- 172.21.15.27:0/1782557184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 msgr2=0x7f34900cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349639c700 1 --2- 172.21.15.27:0/1782557184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f34900cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349639c700 1 -- 172.21.15.27:0/1782557184 >> v1:172.21.15.27:6789/0 conn(0x7f349015c790 legacy=0x7f3490069f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349639c700 1 -- 172.21.15.27:0/1782557184 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490070a50 con 0x7f34900d0220 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f3494b99700 1 -- 172.21.15.27:0/1782557184 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177437233 0 0) 0x7f348c001110 con 0x7f349015c790 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.590+0000 7f349639c700 1 --2- 172.21.15.27:0/1782557184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34900d0220 0x7f34900cd6e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3488009d90 tx=0x7f348800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedf03b7eb619fe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:58.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3494b99700 1 -- 172.21.15.27:0/1782557184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348800e070 con 0x7f34900d0220 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3494b99700 1 -- 172.21.15.27:0/1782557184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348800bb20 con 0x7f34900d0220 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3494b99700 1 -- 172.21.15.27:0/1782557184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488012430 con 0x7f34900d0220 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 -- 172.21.15.27:0/1782557184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34900d0220 msgr2=0x7f34900cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 --2- 172.21.15.27:0/1782557184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34900d0220 0x7f34900cd6e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3488009d90 tx=0x7f348800b330 comp rx=0 tx=0).stop 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 -- 172.21.15.27:0/1782557184 shutdown_connections 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 --2- 172.21.15.27:0/1782557184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34900d0220 0x7f34900cd6e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 --2- 172.21.15.27:0/1782557184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f34900cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 -- 172.21.15.27:0/1782557184 >> 172.21.15.27:0/1782557184 conn(0x7f34900b22f0 msgr2=0x7f34900cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:58.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 -- 172.21.15.27:0/1782557184 shutdown_connections 2023-07-06T20:14:58.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.591+0000 7f3497dff700 1 -- 172.21.15.27:0/1782557184 wait complete. 2023-07-06T20:14:58.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.593+0000 7f3497dff700 1 Processor -- start 2023-07-06T20:14:58.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.593+0000 7f3497dff700 1 -- start start 2023-07-06T20:14:58.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.594+0000 7f3497dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f3490150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.594+0000 7f3497dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900d0220 0x7f34900c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.594+0000 7f3497dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 0x7f34900c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.594+0000 7f3497dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34900c7120 con 0x7f349015c790 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.594+0000 7f3497dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34900b58e0 con 0x7f34900d0220 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.594+0000 7f3497dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34900c8910 con 0x7f349006a670 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 0x7f34900c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 0x7f34900c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59424/0 (socket says 172.21.15.27:59424) 2023-07-06T20:14:58.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 -- 172.21.15.27:0/1651235868 learned_addr learned my addr 172.21.15.27:0/1651235868 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f3495b9b700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f3490150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349539a700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900d0220 0x7f34900c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 -- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 msgr2=0x7f3490150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f3490150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 -- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900d0220 msgr2=0x7f34900c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900d0220 0x7f34900c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349639c700 1 -- 172.21.15.27:0/1651235868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009a70 con 0x7f349015c790 2023-07-06T20:14:58.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.595+0000 7f349539a700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900d0220 0x7f34900c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:58.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.596+0000 7f349639c700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 0x7f34900c83d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f348800b680 tx=0x7f3488004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:58.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.596+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488012430 con 0x7f349015c790 2023-07-06T20:14:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.596+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3488020070 con 0x7f349015c790 2023-07-06T20:14:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.596+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488004280 con 0x7f349015c790 2023-07-06T20:14:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.596+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34900c7330 con 0x7f349015c790 2023-07-06T20:14:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.596+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34900c78f0 con 0x7f349015c790 2023-07-06T20:14:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.597+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34900008d0 con 0x7f349015c790 2023-07-06T20:14:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.598+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34880043e0 con 0x7f349015c790 2023-07-06T20:14:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.599+0000 7f34867fc700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f347406ace0 0x7f347406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.600+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3488022070 con 0x7f349015c790 2023-07-06T20:14:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.600+0000 7f3495b9b700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f347406ace0 0x7f347406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.600+0000 7f3495b9b700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f347406ace0 0x7f347406d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f348c000f60 tx=0x7f348c007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.604+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34880bd310 con 0x7f349015c790 2023-07-06T20:14:58.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.827+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3490071b90 con 0x7f349015c790 2023-07-06T20:14:58.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.828+0000 7f34867fc700 1 -- 172.21.15.27:0/1651235868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f348805f080 con 0x7f349015c790 2023-07-06T20:14:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f347406ace0 msgr2=0x7f347406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f347406ace0 0x7f347406d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f348c000f60 tx=0x7f348c007e70 comp rx=0 tx=0).stop 2023-07-06T20:14:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 msgr2=0x7f34900c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 0x7f34900c83d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f348800b680 tx=0x7f3488004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 shutdown_connections 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f347406ace0 0x7f347406d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f349015c790 0x7f34900c83d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f349006a670 0x7f3490150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 --2- 172.21.15.27:0/1651235868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34900d0220 0x7f34900c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 >> 172.21.15.27:0/1651235868 conn(0x7f34900b22f0 msgr2=0x7f34900cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 shutdown_connections 2023-07-06T20:14:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:58.830+0000 7f3497dff700 1 -- 172.21.15.27:0/1651235868 wait complete. 2023-07-06T20:14:58.834 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-07-06T20:14:58.850 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-07-06T20:14:58.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:58.851 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-07-06T20:14:58.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:14:58.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:14:58.944 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T20:14:58.991 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T20:14:58.991 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:14:59.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.194+0000 7f16193ff700 1 Processor -- start 2023-07-06T20:14:59.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.194+0000 7f16193ff700 1 -- start start 2023-07-06T20:14:59.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.195+0000 7f16193ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16140d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.195+0000 7f16193ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f16140b60e0 0x7f16140d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.195+0000 7f16193ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140cf780 con 0x7f161415fca0 2023-07-06T20:14:59.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.195+0000 7f16193ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16140cd490 con 0x7f16140b60e0 2023-07-06T20:14:59.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.195+0000 7f16193ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16140cd770 con 0x7f16140b5c90 2023-07-06T20:14:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f1612ffd700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f161415fca0 0x7f16140d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45428/0 (socket says 172.21.15.27:45428) 2023-07-06T20:14:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f1612ffd700 1 -- 172.21.15.27:0/3850616258 learned_addr learned my addr 172.21.15.27:0/3850616258 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:14:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f16137fe700 1 --2- 172.21.15.27:0/3850616258 >> v2:172.21.15.27:3300/0 conn(0x7f16140b60e0 0x7f16140d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f16127fc700 1 --2- 172.21.15.27:0/3850616258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16140d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f1611ffb700 1 -- 172.21.15.27:0/3850616258 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771430582 0 0) 0x7f16140cf780 con 0x7f161415fca0 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f1611ffb700 1 -- 172.21.15.27:0/3850616258 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15fc0035b0 con 0x7f161415fca0 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f16137fe700 1 -- 172.21.15.27:0/3850616258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 msgr2=0x7f16140d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f16137fe700 1 --2- 172.21.15.27:0/3850616258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16140d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f16137fe700 1 -- 172.21.15.27:0/3850616258 >> v1:172.21.15.27:6789/0 conn(0x7f161415fca0 legacy=0x7f16140d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.196+0000 7f16137fe700 1 -- 172.21.15.27:0/3850616258 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140cda50 con 0x7f16140b60e0 2023-07-06T20:14:59.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16137fe700 1 --2- 172.21.15.27:0/3850616258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16140b60e0 0x7f16140d3280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f160c009d90 tx=0x7f160c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2785fc0efcb7352d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f1611ffb700 1 -- 172.21.15.27:0/3850616258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c00e070 con 0x7f16140b60e0 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f1611ffb700 1 -- 172.21.15.27:0/3850616258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c00bb20 con 0x7f16140b60e0 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f1611ffb700 1 -- 172.21.15.27:0/3850616258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c012430 con 0x7f16140b60e0 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 -- 172.21.15.27:0/3850616258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16140b60e0 msgr2=0x7f16140d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 --2- 172.21.15.27:0/3850616258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16140b60e0 0x7f16140d3280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f160c009d90 tx=0x7f160c00b330 comp rx=0 tx=0).stop 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 -- 172.21.15.27:0/3850616258 shutdown_connections 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 --2- 172.21.15.27:0/3850616258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16140b60e0 0x7f16140d3280 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 --2- 172.21.15.27:0/3850616258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16140d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 -- 172.21.15.27:0/3850616258 >> 172.21.15.27:0/3850616258 conn(0x7f161406c150 msgr2=0x7f161406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 -- 172.21.15.27:0/3850616258 shutdown_connections 2023-07-06T20:14:59.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.197+0000 7f16193ff700 1 -- 172.21.15.27:0/3850616258 wait complete. 2023-07-06T20:14:59.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.199+0000 7f16193ff700 1 Processor -- start 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.199+0000 7f16193ff700 1 -- start start 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.200+0000 7f16193ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16141f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.200+0000 7f16193ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16140b60e0 0x7f1614143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.200+0000 7f16193ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 0x7f16141543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.200+0000 7f16193ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1614153fa0 con 0x7f161415fca0 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.200+0000 7f16193ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f1b10 con 0x7f16140b60e0 2023-07-06T20:14:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.200+0000 7f16193ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1614153320 con 0x7f16140b5c90 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 0x7f16141543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 0x7f16141543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59430/0 (socket says 172.21.15.27:59430) 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 -- 172.21.15.27:0/4169217868 learned_addr learned my addr 172.21.15.27:0/4169217868 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f1612ffd700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16141f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16127fc700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16140b60e0 0x7f1614143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 -- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 msgr2=0x7f16141f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16141f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 -- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16140b60e0 msgr2=0x7f1614143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16140b60e0 0x7f1614143f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 -- 172.21.15.27:0/4169217868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c009a70 con 0x7f161415fca0 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.201+0000 7f16137fe700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 0x7f16141543c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f160c014040 tx=0x7f160c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.202+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c012430 con 0x7f161415fca0 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.202+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c020070 con 0x7f161415fca0 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.202+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c003ea0 con 0x7f161415fca0 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.202+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1614153600 con 0x7f161415fca0 2023-07-06T20:14:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.202+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1614153bc0 con 0x7f161415fca0 2023-07-06T20:14:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.203+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16141540e0 con 0x7f161415fca0 2023-07-06T20:14:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.204+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f160c004040 con 0x7f161415fca0 2023-07-06T20:14:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.205+0000 7f15fb7fe700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f160006ace0 0x7f160006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.205+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f160c022070 con 0x7f161415fca0 2023-07-06T20:14:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.206+0000 7f1612ffd700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f160006ace0 0x7f160006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.206+0000 7f1612ffd700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f160006ace0 0x7f160006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1604007fd0 tx=0x7f1604007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.209+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16141540e0 con 0x7f161415fca0 2023-07-06T20:14:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.459+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16141540e0 con 0x7f161415fca0 2023-07-06T20:14:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.460+0000 7f15fb7fe700 1 -- 172.21.15.27:0/4169217868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f16141540e0 con 0x7f161415fca0 2023-07-06T20:14:59.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:59.462 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:14:59.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.462+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f160006ace0 msgr2=0x7f160006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.462+0000 7f16193ff700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f160006ace0 0x7f160006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1604007fd0 tx=0x7f1604007dc0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.462+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 msgr2=0x7f16141543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.462+0000 7f16193ff700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 0x7f16141543c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f160c014040 tx=0x7f160c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 shutdown_connections 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f160006ace0 0x7f160006d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f161415fca0 0x7f16141543c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16140b5c90 0x7f16141f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 --2- 172.21.15.27:0/4169217868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16140b60e0 0x7f1614143f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 >> 172.21.15.27:0/4169217868 conn(0x7f161406c150 msgr2=0x7f16140b87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 shutdown_connections 2023-07-06T20:14:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.463+0000 7f16193ff700 1 -- 172.21.15.27:0/4169217868 wait complete. 2023-07-06T20:14:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:14:59.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:14:59.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.685+0000 7fe935bff700 1 Processor -- start 2023-07-06T20:14:59.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.685+0000 7fe935bff700 1 -- start start 2023-07-06T20:14:59.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe935bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe935bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe935bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9300cf780 con 0x7fe9300b6110 2023-07-06T20:14:59.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe935bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9300cd490 con 0x7fe93015fca0 2023-07-06T20:14:59.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe935bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9300cd770 con 0x7fe9300b5c90 2023-07-06T20:14:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe92f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe92f7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59436/0 (socket says 172.21.15.27:59436) 2023-07-06T20:14:59.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.686+0000 7fe92f7fe700 1 -- 172.21.15.27:0/3681095107 learned_addr learned my addr 172.21.15.27:0/3681095107 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:59.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92effd700 1 --2- 172.21.15.27:0/3681095107 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92f7fe700 1 -- 172.21.15.27:0/3681095107 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 msgr2=0x7fe9300d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92f7fe700 1 --2- 172.21.15.27:0/3681095107 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92f7fe700 1 -- 172.21.15.27:0/3681095107 >> v1:172.21.15.153:6789/0 conn(0x7fe9300b6110 legacy=0x7fe9300cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92f7fe700 1 -- 172.21.15.27:0/3681095107 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9300cda50 con 0x7fe93015fca0 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92effd700 1 --2- 172.21.15.27:0/3681095107 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92f7fe700 1 --2- 172.21.15.27:0/3681095107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe9300d1420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe93006b6f0 tx=0x7fe920009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e741dcf00d1c6cb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92e7fc700 1 -- 172.21.15.27:0/3681095107 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920004230 con 0x7fe93015fca0 2023-07-06T20:14:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92e7fc700 1 -- 172.21.15.27:0/3681095107 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe920004390 con 0x7fe93015fca0 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.687+0000 7fe92e7fc700 1 -- 172.21.15.27:0/3681095107 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200036a0 con 0x7fe93015fca0 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 -- 172.21.15.27:0/3681095107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 msgr2=0x7fe9300d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 --2- 172.21.15.27:0/3681095107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe9300d1420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe93006b6f0 tx=0x7fe920009bd0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 -- 172.21.15.27:0/3681095107 shutdown_connections 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 --2- 172.21.15.27:0/3681095107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe9300d1420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 --2- 172.21.15.27:0/3681095107 >> v2:172.21.15.153:3300/0 conn(0x7fe9300b5c90 0x7fe9300d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 -- 172.21.15.27:0/3681095107 >> 172.21.15.27:0/3681095107 conn(0x7fe93006c150 msgr2=0x7fe93006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 -- 172.21.15.27:0/3681095107 shutdown_connections 2023-07-06T20:14:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.688+0000 7fe935bff700 1 -- 172.21.15.27:0/3681095107 wait complete. 2023-07-06T20:14:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.690+0000 7fe935bff700 1 Processor -- start 2023-07-06T20:14:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.690+0000 7fe935bff700 1 -- start start 2023-07-06T20:14:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe935bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b5c90 0x7fe9301ef410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe935bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9300b6110 0x7fe93015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe935bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe93015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe935bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93015e4d0 con 0x7fe93015fca0 2023-07-06T20:14:59.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe935bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93015dc10 con 0x7fe9300b5c90 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe935bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93015de80 con 0x7fe9300b6110 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe92ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe93015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe92ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe93015bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59438/0 (socket says 172.21.15.27:59438) 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe92ffff700 1 -- 172.21.15.27:0/2289342251 learned_addr learned my addr 172.21.15.27:0/2289342251 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe92f7fe700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b5c90 0x7fe9301ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.691+0000 7fe92effd700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9300b6110 0x7fe93015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92ffff700 1 -- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9300b6110 msgr2=0x7fe93015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92ffff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9300b6110 0x7fe93015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92ffff700 1 -- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b5c90 msgr2=0x7fe9301ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92ffff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b5c90 0x7fe9301ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92ffff700 1 -- 172.21.15.27:0/2289342251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920009550 con 0x7fe93015fca0 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92effd700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9300b6110 0x7fe93015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe92ffff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe93015bd70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe92800cc40 tx=0x7fe92800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92800ab30 con 0x7fe93015fca0 2023-07-06T20:14:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92800ac90 con 0x7fe93015fca0 2023-07-06T20:14:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93015e160 con 0x7fe93015fca0 2023-07-06T20:14:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928012420 con 0x7fe93015fca0 2023-07-06T20:14:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.692+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93015e960 con 0x7fe93015fca0 2023-07-06T20:14:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.693+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93015e680 con 0x7fe93015fca0 2023-07-06T20:14:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.694+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe928012580 con 0x7fe93015fca0 2023-07-06T20:14:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.696+0000 7fe917fff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91c06ace0 0x7fe91c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.696+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe928089ca0 con 0x7fe93015fca0 2023-07-06T20:14:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.696+0000 7fe92f7fe700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91c06ace0 0x7fe91c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.697+0000 7fe92f7fe700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91c06ace0 0x7fe91c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe920009130 tx=0x7fe920008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.701+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe928054d20 con 0x7fe93015fca0 2023-07-06T20:14:59.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.911+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9300008d0 con 0x7fe93015fca0 2023-07-06T20:14:59.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.912+0000 7fe917fff700 1 -- 172.21.15.27:0/2289342251 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7fe928016020 con 0x7fe93015fca0 2023-07-06T20:14:59.914 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:14:59.915 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:14:59.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.914+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91c06ace0 msgr2=0x7fe91c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.914+0000 7fe935bff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91c06ace0 0x7fe91c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe920009130 tx=0x7fe920008030 comp rx=0 tx=0).stop 2023-07-06T20:14:59.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.914+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 msgr2=0x7fe93015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:59.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.914+0000 7fe935bff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe93015bd70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe92800cc40 tx=0x7fe92800cf50 comp rx=0 tx=0).stop 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 shutdown_connections 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe91c06ace0 0x7fe91c06d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe93015fca0 0x7fe93015bd70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9300b5c90 0x7fe9301ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 --2- 172.21.15.27:0/2289342251 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe9300b6110 0x7fe93015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 >> 172.21.15.27:0/2289342251 conn(0x7fe93006c150 msgr2=0x7fe93015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:59.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 shutdown_connections 2023-07-06T20:14:59.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:14:59.915+0000 7fe935bff700 1 -- 172.21.15.27:0/2289342251 wait complete. 2023-07-06T20:14:59.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:13:57.143873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:13:58.519309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:14:59.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:15:00.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.141+0000 7f50559ff700 1 Processor -- start 2023-07-06T20:15:00.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.142+0000 7f50559ff700 1 -- start start 2023-07-06T20:15:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f50559ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f505015fca0 0x7f50500d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f50559ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50500b5c90 0x7f50500d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f50559ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500cf780 con 0x7f50500b6110 2023-07-06T20:15:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f50559ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50500cd490 con 0x7f505015fca0 2023-07-06T20:15:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f50559ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50500cd770 con 0x7f50500b5c90 2023-07-06T20:15:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f504effd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f505015fca0 0x7f50500d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f504effd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f505015fca0 0x7f50500d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59448/0 (socket says 172.21.15.27:59448) 2023-07-06T20:15:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.143+0000 7f504effd700 1 -- 172.21.15.27:0/1524121768 learned_addr learned my addr 172.21.15.27:0/1524121768 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504effd700 1 -- 172.21.15.27:0/1524121768 >> v2:172.21.15.153:3300/0 conn(0x7f50500b5c90 msgr2=0x7f50500d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504effd700 1 --2- 172.21.15.27:0/1524121768 >> v2:172.21.15.153:3300/0 conn(0x7f50500b5c90 0x7f50500d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504effd700 1 -- 172.21.15.27:0/1524121768 >> v1:172.21.15.153:6789/0 conn(0x7f50500b6110 legacy=0x7f50500cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504effd700 1 -- 172.21.15.27:0/1524121768 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500cda50 con 0x7f505015fca0 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504effd700 1 --2- 172.21.15.27:0/1524121768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f505015fca0 0x7f50500d1420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f505006b6f0 tx=0x7f5040009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94376a69337b2293 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504dffb700 1 -- 172.21.15.27:0/1524121768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040004230 con 0x7f505015fca0 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504dffb700 1 -- 172.21.15.27:0/1524121768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5040004390 con 0x7f505015fca0 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.144+0000 7f504dffb700 1 -- 172.21.15.27:0/1524121768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400036a0 con 0x7f505015fca0 2023-07-06T20:15:00.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 -- 172.21.15.27:0/1524121768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f505015fca0 msgr2=0x7f50500d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 --2- 172.21.15.27:0/1524121768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f505015fca0 0x7f50500d1420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f505006b6f0 tx=0x7f5040009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 -- 172.21.15.27:0/1524121768 shutdown_connections 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 --2- 172.21.15.27:0/1524121768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f505015fca0 0x7f50500d1420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 --2- 172.21.15.27:0/1524121768 >> v2:172.21.15.153:3300/0 conn(0x7f50500b5c90 0x7f50500d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 -- 172.21.15.27:0/1524121768 >> 172.21.15.27:0/1524121768 conn(0x7f505006c150 msgr2=0x7f505006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 -- 172.21.15.27:0/1524121768 shutdown_connections 2023-07-06T20:15:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.145+0000 7f50559ff700 1 -- 172.21.15.27:0/1524121768 wait complete. 2023-07-06T20:15:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.147+0000 7f50559ff700 1 Processor -- start 2023-07-06T20:15:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.147+0000 7f50559ff700 1 -- start start 2023-07-06T20:15:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f50559ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50500b5c90 0x7f5050158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f50559ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 0x7f50501f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f50559ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505015fca0 0x7f50501f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f50559ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5050158b90 con 0x7f50500b6110 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f50559ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50501f54f0 con 0x7f50500b5c90 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f50559ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50501f2440 con 0x7f505015fca0 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 0x7f50501f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 0x7f50501f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59464/0 (socket says 172.21.15.27:59464) 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 -- 172.21.15.27:0/1613998517 learned_addr learned my addr 172.21.15.27:0/1613998517 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 -- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505015fca0 msgr2=0x7f50501f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505015fca0 0x7f50501f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 -- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50500b5c90 msgr2=0x7f5050158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50500b5c90 0x7f5050158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.148+0000 7f504e7fc700 1 -- 172.21.15.27:0/1613998517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009550 con 0x7f50500b6110 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.149+0000 7f504e7fc700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 0x7f50501f1f00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f504400b530 tx=0x7f504400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.149+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044011830 con 0x7f50500b6110 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.149+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50440119b0 con 0x7f50500b6110 2023-07-06T20:15:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.149+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50501f26e0 con 0x7f50500b6110 2023-07-06T20:15:00.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.149+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044011d60 con 0x7f50500b6110 2023-07-06T20:15:00.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.149+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50501f2ca0 con 0x7f50500b6110 2023-07-06T20:15:00.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.150+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50500008d0 con 0x7f50500b6110 2023-07-06T20:15:00.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.151+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f504401a400 con 0x7f50500b6110 2023-07-06T20:15:00.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.152+0000 7f5037fff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503c06ace0 0x7f503c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.152+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f50440895c0 con 0x7f50500b6110 2023-07-06T20:15:00.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.153+0000 7f504effd700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503c06ace0 0x7f503c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.154+0000 7f504effd700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503c06ace0 0x7f503c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50500cc8b0 tx=0x7f5040010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:00.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.157+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5044054630 con 0x7f50500b6110 2023-07-06T20:15:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.364+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50501f2e60 con 0x7f50500b6110 2023-07-06T20:15:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.365+0000 7f5037fff700 1 -- 172.21.15.27:0/1613998517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5044057c50 con 0x7f50500b6110 2023-07-06T20:15:00.367 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:15:00.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503c06ace0 msgr2=0x7f503c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503c06ace0 0x7f503c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50500cc8b0 tx=0x7f5040010000 comp rx=0 tx=0).stop 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 msgr2=0x7f50501f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 0x7f50501f1f00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f504400b530 tx=0x7f504400b8f0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 shutdown_connections 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f503c06ace0 0x7f503c06d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f505015fca0 0x7f50501f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50500b5c90 0x7f5050158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 --2- 172.21.15.27:0/1613998517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50500b6110 0x7f50501f1f00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 >> 172.21.15.27:0/1613998517 conn(0x7f505006c150 msgr2=0x7f50500d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:00.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 shutdown_connections 2023-07-06T20:15:00.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.367+0000 7f50559ff700 1 -- 172.21.15.27:0/1613998517 wait complete. 2023-07-06T20:15:00.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:15:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.592+0000 7fc778fff700 1 Processor -- start 2023-07-06T20:15:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.592+0000 7fc778fff700 1 -- start start 2023-07-06T20:15:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.593+0000 7fc778fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc77406c330 0x7fc77406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.593+0000 7fc778fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc7740c9480 0x7fc774156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.593+0000 7fc778fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77406cc40 con 0x7fc7740c9010 2023-07-06T20:15:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.593+0000 7fc778fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7740c9a60 con 0x7fc7740c9480 2023-07-06T20:15:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.593+0000 7fc778fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc774159b70 con 0x7fc77406c330 2023-07-06T20:15:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc77406c330 0x7fc77406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc77406c330 0x7fc77406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:57924/0 (socket says 172.21.15.27:57924) 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 -- 172.21.15.27:0/3552557048 learned_addr learned my addr 172.21.15.27:0/3552557048 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc772d9d700 1 --2- 172.21.15.27:0/3552557048 >> v2:172.21.15.27:3300/0 conn(0x7fc7740c9480 0x7fc774156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 -- 172.21.15.27:0/3552557048 >> v2:172.21.15.27:3300/0 conn(0x7fc7740c9480 msgr2=0x7fc774156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 --2- 172.21.15.27:0/3552557048 >> v2:172.21.15.27:3300/0 conn(0x7fc7740c9480 0x7fc774156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 -- 172.21.15.27:0/3552557048 >> v1:172.21.15.153:6789/0 conn(0x7fc7740c9010 legacy=0x7fc7741565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 -- 172.21.15.27:0/3552557048 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774159e50 con 0x7fc77406c330 2023-07-06T20:15:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.594+0000 7fc77259c700 1 --2- 172.21.15.27:0/3552557048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77406c330 0x7fc77406c700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc77406f160 tx=0x7fc764009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad53257ea63b510b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc77159a700 1 -- 172.21.15.27:0/3552557048 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764011830 con 0x7fc77406c330 2023-07-06T20:15:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc77159a700 1 -- 172.21.15.27:0/3552557048 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764011990 con 0x7fc77406c330 2023-07-06T20:15:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc77159a700 1 -- 172.21.15.27:0/3552557048 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764011d20 con 0x7fc77406c330 2023-07-06T20:15:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 -- 172.21.15.27:0/3552557048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77406c330 msgr2=0x7fc77406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 --2- 172.21.15.27:0/3552557048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77406c330 0x7fc77406c700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc77406f160 tx=0x7fc764009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 -- 172.21.15.27:0/3552557048 shutdown_connections 2023-07-06T20:15:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 --2- 172.21.15.27:0/3552557048 >> v2:172.21.15.27:3300/0 conn(0x7fc7740c9480 0x7fc774156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 --2- 172.21.15.27:0/3552557048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77406c330 0x7fc77406c700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 -- 172.21.15.27:0/3552557048 >> 172.21.15.27:0/3552557048 conn(0x7fc7740cb440 msgr2=0x7fc7740cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 -- 172.21.15.27:0/3552557048 shutdown_connections 2023-07-06T20:15:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.595+0000 7fc778fff700 1 -- 172.21.15.27:0/3552557048 wait complete. 2023-07-06T20:15:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.597+0000 7fc778fff700 1 Processor -- start 2023-07-06T20:15:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.597+0000 7fc778fff700 1 -- start start 2023-07-06T20:15:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc778fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 0x7fc7740ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc778fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7740c9010 0x7fc7740b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc778fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7740c9480 0x7fc7740aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc778fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741f0610 con 0x7fc77406c330 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc778fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740aeb00 con 0x7fc7740c9010 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc778fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740c7640 con 0x7fc7740c9480 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc77259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 0x7fc7740ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc77259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 0x7fc7740ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59470/0 (socket says 172.21.15.27:59470) 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.598+0000 7fc77259c700 1 -- 172.21.15.27:0/755165717 learned_addr learned my addr 172.21.15.27:0/755165717 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc772d9d700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7740c9480 0x7fc7740aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc771d9b700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7740c9010 0x7fc7740b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc77259c700 1 -- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7740c9480 msgr2=0x7fc7740aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc77259c700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7740c9480 0x7fc7740aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc77259c700 1 -- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7740c9010 msgr2=0x7fc7740b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc77259c700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7740c9010 0x7fc7740b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc77259c700 1 -- 172.21.15.27:0/755165717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764009550 con 0x7fc77406c330 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc771d9b700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7740c9010 0x7fc7740b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc77259c700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 0x7fc7740ae5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc764009c10 tx=0x7fc764004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640036a0 con 0x7fc77406c330 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76401e070 con 0x7fc77406c330 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7741f08b0 con 0x7fc77406c330 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.599+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764003a60 con 0x7fc77406c330 2023-07-06T20:15:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.600+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7740c7920 con 0x7fc77406c330 2023-07-06T20:15:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.601+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7740008d0 con 0x7fc77406c330 2023-07-06T20:15:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.602+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc764003bc0 con 0x7fc77406c330 2023-07-06T20:15:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.602+0000 7fc762ffd700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc75406ace0 0x7fc75406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.603+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc76408a3c0 con 0x7fc77406c330 2023-07-06T20:15:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.603+0000 7fc771d9b700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc75406ace0 0x7fc75406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.604+0000 7fc771d9b700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc75406ace0 0x7fc75406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc7740c5400 tx=0x7fc768009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.607+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc764055420 con 0x7fc77406c330 2023-07-06T20:15:00.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.811+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7740ae100 con 0x7fc77406c330 2023-07-06T20:15:00.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.812+0000 7fc762ffd700 1 -- 172.21.15.27:0/755165717 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc764058a40 con 0x7fc77406c330 2023-07-06T20:15:00.814 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:15:00.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc75406ace0 msgr2=0x7fc75406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc75406ace0 0x7fc75406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc7740c5400 tx=0x7fc768009040 comp rx=0 tx=0).stop 2023-07-06T20:15:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 msgr2=0x7fc7740ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 0x7fc7740ae5c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc764009c10 tx=0x7fc764004750 comp rx=0 tx=0).stop 2023-07-06T20:15:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 shutdown_connections 2023-07-06T20:15:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc75406ace0 0x7fc75406d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc7740c9480 0x7fc7740aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77406c330 0x7fc7740ae5c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 --2- 172.21.15.27:0/755165717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7740c9010 0x7fc7740b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 >> 172.21.15.27:0/755165717 conn(0x7fc7740cb440 msgr2=0x7fc77415cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.814+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 shutdown_connections 2023-07-06T20:15:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:00.815+0000 7fc778fff700 1 -- 172.21.15.27:0/755165717 wait complete. 2023-07-06T20:15:00.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:15:01.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.039+0000 7f38c79ff700 1 Processor -- start 2023-07-06T20:15:01.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.039+0000 7f38c79ff700 1 -- start start 2023-07-06T20:15:01.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c79ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f38c015fca0 0x7f38c00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c79ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 0x7f38c00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c79ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c00cf780 con 0x7f38c00b60e0 2023-07-06T20:15:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c79ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c00cd490 con 0x7f38c015fca0 2023-07-06T20:15:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c79ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00cd770 con 0x7f38c00b5c90 2023-07-06T20:15:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c4f9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 0x7f38c00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.040+0000 7f38c579b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f38c015fca0 0x7f38c00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 0x7f38c00d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:42642/0 (socket says 172.21.15.27:42642) 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 -- 172.21.15.27:0/3984507605 learned_addr learned my addr 172.21.15.27:0/3984507605 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c5f9c700 1 --1- 172.21.15.27:0/3984507605 >> v1:172.21.15.153:6789/0 conn(0x7f38c00b60e0 0x7f38c00cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:46940/0 (socket says 172.21.15.27:46940) 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38b7fff700 1 -- 172.21.15.27:0/3984507605 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763122178 0 0) 0x7f38c00cf780 con 0x7f38c00b60e0 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38b7fff700 1 -- 172.21.15.27:0/3984507605 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38ac0035b0 con 0x7f38c00b60e0 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 -- 172.21.15.27:0/3984507605 >> v2:172.21.15.153:3300/0 conn(0x7f38c015fca0 msgr2=0x7f38c00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 --2- 172.21.15.27:0/3984507605 >> v2:172.21.15.153:3300/0 conn(0x7f38c015fca0 0x7f38c00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 -- 172.21.15.27:0/3984507605 >> v1:172.21.15.153:6789/0 conn(0x7f38c00b60e0 legacy=0x7f38c00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 -- 172.21.15.27:0/3984507605 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00cda50 con 0x7f38c00b5c90 2023-07-06T20:15:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c579b700 1 --2- 172.21.15.27:0/3984507605 >> v2:172.21.15.153:3300/0 conn(0x7f38c015fca0 0x7f38c00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.041+0000 7f38c4f9a700 1 --2- 172.21.15.27:0/3984507605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 0x7f38c00d2d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38b00098f0 tx=0x7f38b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6f06b59287d4ed server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38b7fff700 1 -- 172.21.15.27:0/3984507605 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0011830 con 0x7f38c00b5c90 2023-07-06T20:15:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38b7fff700 1 -- 172.21.15.27:0/3984507605 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b0011990 con 0x7f38c00b5c90 2023-07-06T20:15:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38b7fff700 1 -- 172.21.15.27:0/3984507605 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0011d20 con 0x7f38c00b5c90 2023-07-06T20:15:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 -- 172.21.15.27:0/3984507605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 msgr2=0x7f38c00d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 --2- 172.21.15.27:0/3984507605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 0x7f38c00d2d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38b00098f0 tx=0x7f38b0009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 -- 172.21.15.27:0/3984507605 shutdown_connections 2023-07-06T20:15:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 --2- 172.21.15.27:0/3984507605 >> v2:172.21.15.153:3300/0 conn(0x7f38c015fca0 0x7f38c00d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 --2- 172.21.15.27:0/3984507605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c00b5c90 0x7f38c00d2d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 -- 172.21.15.27:0/3984507605 >> 172.21.15.27:0/3984507605 conn(0x7f38c006c150 msgr2=0x7f38c006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 -- 172.21.15.27:0/3984507605 shutdown_connections 2023-07-06T20:15:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.042+0000 7f38c79ff700 1 -- 172.21.15.27:0/3984507605 wait complete. 2023-07-06T20:15:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.044+0000 7f38c79ff700 1 Processor -- start 2023-07-06T20:15:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.044+0000 7f38c79ff700 1 -- start start 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.045+0000 7f38c79ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38c00b5c90 0x7f38c01582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.045+0000 7f38c79ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 0x7f38c015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.045+0000 7f38c79ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c015fca0 0x7f38c015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.045+0000 7f38c79ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b29f0 con 0x7f38c00b60e0 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.045+0000 7f38c79ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b2cd0 con 0x7f38c00b5c90 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.045+0000 7f38c79ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b2fb0 con 0x7f38c015fca0 2023-07-06T20:15:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 0x7f38c015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 0x7f38c015b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59484/0 (socket says 172.21.15.27:59484) 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 -- 172.21.15.27:0/1330887129 learned_addr learned my addr 172.21.15.27:0/1330887129 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c5f9c700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c015fca0 0x7f38c015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c579b700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38c00b5c90 0x7f38c01582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 -- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c015fca0 msgr2=0x7f38c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c015fca0 0x7f38c015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 -- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38c00b5c90 msgr2=0x7f38c01582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38c00b5c90 0x7f38c01582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 -- 172.21.15.27:0/1330887129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0094a0 con 0x7f38c00b60e0 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.046+0000 7f38c4f9a700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 0x7f38c015b7c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.047+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0004480 con 0x7f38c00b60e0 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.047+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b001f070 con 0x7f38c00b60e0 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.047+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b0009550 con 0x7f38c00b60e0 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.047+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00038c0 con 0x7f38c00b60e0 2023-07-06T20:15:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.047+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c01f2de0 con 0x7f38c00b60e0 2023-07-06T20:15:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.048+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c00008d0 con 0x7f38c00b60e0 2023-07-06T20:15:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.049+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38b0021030 con 0x7f38c00b60e0 2023-07-06T20:15:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.050+0000 7f38b5ffb700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38a406ace0 0x7f38a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.050+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f38b008a930 con 0x7f38c00b60e0 2023-07-06T20:15:01.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.051+0000 7f38c579b700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38a406ace0 0x7f38a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.051+0000 7f38c579b700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38a406ace0 0x7f38a406d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009c60 tx=0x7f38bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.057+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38b0055990 con 0x7f38c00b60e0 2023-07-06T20:15:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.263+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38c00b2b30 con 0x7f38c00b60e0 2023-07-06T20:15:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.263+0000 7f38b5ffb700 1 -- 172.21.15.27:0/1330887129 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38b0015020 con 0x7f38c00b60e0 2023-07-06T20:15:01.266 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:15:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38a406ace0 msgr2=0x7f38a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38a406ace0 0x7f38a406d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009c60 tx=0x7f38bc008040 comp rx=0 tx=0).stop 2023-07-06T20:15:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 msgr2=0x7f38c015b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 0x7f38c015b7c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0004750 comp rx=0 tx=0).stop 2023-07-06T20:15:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 shutdown_connections 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f38a406ace0 0x7f38a406d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38c015fca0 0x7f38c015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38c00b5c90 0x7f38c01582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 --2- 172.21.15.27:0/1330887129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00b60e0 0x7f38c015b7c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 >> 172.21.15.27:0/1330887129 conn(0x7f38c006c150 msgr2=0x7f38c00cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 shutdown_connections 2023-07-06T20:15:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.266+0000 7f38c79ff700 1 -- 172.21.15.27:0/1330887129 wait complete. 2023-07-06T20:15:01.285 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:15:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.490+0000 7f2364bff700 1 Processor -- start 2023-07-06T20:15:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.490+0000 7f2364bff700 1 -- start start 2023-07-06T20:15:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.491+0000 7f2364bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f23600b5c90 0x7f23600d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.491+0000 7f2364bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23600b6110 0x7f23600d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.491+0000 7f2364bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23600cf780 con 0x7f236015fca0 2023-07-06T20:15:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.491+0000 7f2364bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23600cd490 con 0x7f23600b5c90 2023-07-06T20:15:01.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.491+0000 7f2364bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600cd770 con 0x7f23600b6110 2023-07-06T20:15:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f23600b5c90 0x7f23600d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235ed9d700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23600b6110 0x7f23600d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235e59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f236015fca0 0x7f23600d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:46954/0 (socket says 172.21.15.27:46954) 2023-07-06T20:15:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235e59c700 1 -- 172.21.15.27:0/1098536547 learned_addr learned my addr 172.21.15.27:0/1098536547 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235d59a700 1 -- 172.21.15.27:0/1098536547 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005388479 0 0) 0x7f23600cf780 con 0x7f236015fca0 2023-07-06T20:15:01.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235d59a700 1 -- 172.21.15.27:0/1098536547 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23480035b0 con 0x7f236015fca0 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235dd9b700 1 -- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23600b6110 msgr2=0x7f23600d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235dd9b700 1 --2- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23600b6110 0x7f23600d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235dd9b700 1 -- 172.21.15.27:0/1098536547 >> v1:172.21.15.153:6789/0 conn(0x7f236015fca0 legacy=0x7f23600d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235dd9b700 1 -- 172.21.15.27:0/1098536547 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600cda50 con 0x7f23600b5c90 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.492+0000 7f235ed9d700 1 --2- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23600b6110 0x7f23600d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.493+0000 7f235dd9b700 1 --2- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23600d2d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2354009840 tx=0x7f2354009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=519a4b198b091c2c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.493+0000 7f235d59a700 1 -- 172.21.15.27:0/1098536547 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354011830 con 0x7f23600b5c90 2023-07-06T20:15:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.493+0000 7f235d59a700 1 -- 172.21.15.27:0/1098536547 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2354011990 con 0x7f23600b5c90 2023-07-06T20:15:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.493+0000 7f235d59a700 1 -- 172.21.15.27:0/1098536547 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354011d20 con 0x7f23600b5c90 2023-07-06T20:15:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 -- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 msgr2=0x7f23600d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 --2- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23600d2d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2354009840 tx=0x7f2354009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 -- 172.21.15.27:0/1098536547 shutdown_connections 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 --2- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f23600b6110 0x7f23600d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 --2- 172.21.15.27:0/1098536547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23600d2d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 -- 172.21.15.27:0/1098536547 >> 172.21.15.27:0/1098536547 conn(0x7f236006c150 msgr2=0x7f236006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 -- 172.21.15.27:0/1098536547 shutdown_connections 2023-07-06T20:15:01.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.494+0000 7f2364bff700 1 -- 172.21.15.27:0/1098536547 wait complete. 2023-07-06T20:15:01.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.496+0000 7f2364bff700 1 Processor -- start 2023-07-06T20:15:01.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.496+0000 7f2364bff700 1 -- start start 2023-07-06T20:15:01.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f2364bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23601f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f2364bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 0x7f23600c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f2364bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f236015fca0 0x7f23600cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f2364bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600cb1c0 con 0x7f23600b6110 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f2364bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600ca5b0 con 0x7f23600b5c90 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f2364bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600ca890 con 0x7f236015fca0 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f235dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 0x7f23600c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f235dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 0x7f23600c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59488/0 (socket says 172.21.15.27:59488) 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f235dd9b700 1 -- 172.21.15.27:0/1585930341 learned_addr learned my addr 172.21.15.27:0/1585930341 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f235e59c700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23601f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.497+0000 7f235ed9d700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f236015fca0 0x7f23600cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f235dd9b700 1 -- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f236015fca0 msgr2=0x7f23600cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f235dd9b700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f236015fca0 0x7f23600cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f235dd9b700 1 -- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 msgr2=0x7f23601f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f235dd9b700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23601f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f235dd9b700 1 -- 172.21.15.27:0/1585930341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009a70 con 0x7f23600b6110 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f235dd9b700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 0x7f23600c9f10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23540090b0 tx=0x7f2354004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354004550 con 0x7f23600b6110 2023-07-06T20:15:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235401f070 con 0x7f23600b6110 2023-07-06T20:15:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540038e0 con 0x7f23600b6110 2023-07-06T20:15:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23540094a0 con 0x7f23600b6110 2023-07-06T20:15:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.498+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23600cae50 con 0x7f23600b6110 2023-07-06T20:15:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.499+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23600c7970 con 0x7f23600b6110 2023-07-06T20:15:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.500+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2354003a40 con 0x7f23600b6110 2023-07-06T20:15:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.501+0000 7f234effd700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f234006ace0 0x7f234006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.501+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2354021070 con 0x7f23600b6110 2023-07-06T20:15:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.502+0000 7f235e59c700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f234006ace0 0x7f234006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.502+0000 7f235e59c700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f234006ace0 0x7f234006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2350007fd0 tx=0x7f2350007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.506+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f235408ce60 con 0x7f23600b6110 2023-07-06T20:15:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.712+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23600008d0 con 0x7f23600b6110 2023-07-06T20:15:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.713+0000 7f234effd700 1 -- 172.21.15.27:0/1585930341 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2354055d00 con 0x7f23600b6110 2023-07-06T20:15:01.715 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:15:01.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f234006ace0 msgr2=0x7f234006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f234006ace0 0x7f234006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2350007fd0 tx=0x7f2350007dc0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 msgr2=0x7f23600c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 0x7f23600c9f10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23540090b0 tx=0x7f2354004750 comp rx=0 tx=0).stop 2023-07-06T20:15:01.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 shutdown_connections 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f234006ace0 0x7f234006d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f236015fca0 0x7f23600cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23600b5c90 0x7f23601f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 --2- 172.21.15.27:0/1585930341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600b6110 0x7f23600c9f10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 >> 172.21.15.27:0/1585930341 conn(0x7f236006c150 msgr2=0x7f23600d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 shutdown_connections 2023-07-06T20:15:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.715+0000 7f2364bff700 1 -- 172.21.15.27:0/1585930341 wait complete. 2023-07-06T20:15:01.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:01.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.937+0000 7fa4993ff700 1 Processor -- start 2023-07-06T20:15:01.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.937+0000 7fa4993ff700 1 -- start start 2023-07-06T20:15:01.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.938+0000 7fa4993ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa49415fca0 0x7fa4940d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.938+0000 7fa4993ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4940b5c90 0x7fa4940d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.938+0000 7fa4993ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4940cf780 con 0x7fa4940b6110 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.938+0000 7fa4993ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4940cd490 con 0x7fa49415fca0 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.938+0000 7fa4993ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4940cd770 con 0x7fa4940b5c90 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa49415fca0 0x7fa4940d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa49415fca0 0x7fa4940d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59494/0 (socket says 172.21.15.27:59494) 2023-07-06T20:15:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 -- 172.21.15.27:0/2278239446 learned_addr learned my addr 172.21.15.27:0/2278239446 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:01.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa48bfff700 1 --2- 172.21.15.27:0/2278239446 >> v2:172.21.15.153:3300/0 conn(0x7fa4940b5c90 0x7fa4940d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 -- 172.21.15.27:0/2278239446 >> v2:172.21.15.153:3300/0 conn(0x7fa4940b5c90 msgr2=0x7fa4940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:01.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 --2- 172.21.15.27:0/2278239446 >> v2:172.21.15.153:3300/0 conn(0x7fa4940b5c90 0x7fa4940d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 -- 172.21.15.27:0/2278239446 >> v1:172.21.15.153:6789/0 conn(0x7fa4940b6110 legacy=0x7fa4940cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.939+0000 7fa492ffd700 1 -- 172.21.15.27:0/2278239446 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4940cda50 con 0x7fa49415fca0 2023-07-06T20:15:01.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa492ffd700 1 --2- 172.21.15.27:0/2278239446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa4940d1420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa49406b6f0 tx=0x7fa484009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2b02e9a6ef0160 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4927fc700 1 -- 172.21.15.27:0/2278239446 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48400b870 con 0x7fa49415fca0 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4927fc700 1 -- 172.21.15.27:0/2278239446 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48400b9d0 con 0x7fa49415fca0 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4927fc700 1 -- 172.21.15.27:0/2278239446 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48400bd60 con 0x7fa49415fca0 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4993ff700 1 -- 172.21.15.27:0/2278239446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 msgr2=0x7fa4940d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4993ff700 1 --2- 172.21.15.27:0/2278239446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa4940d1420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa49406b6f0 tx=0x7fa484009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4993ff700 1 -- 172.21.15.27:0/2278239446 shutdown_connections 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4993ff700 1 --2- 172.21.15.27:0/2278239446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa4940d1420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4993ff700 1 --2- 172.21.15.27:0/2278239446 >> v2:172.21.15.153:3300/0 conn(0x7fa4940b5c90 0x7fa4940d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.940+0000 7fa4993ff700 1 -- 172.21.15.27:0/2278239446 >> 172.21.15.27:0/2278239446 conn(0x7fa49406c150 msgr2=0x7fa49406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.941+0000 7fa4993ff700 1 -- 172.21.15.27:0/2278239446 shutdown_connections 2023-07-06T20:15:01.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.941+0000 7fa4993ff700 1 -- 172.21.15.27:0/2278239446 wait complete. 2023-07-06T20:15:01.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.942+0000 7fa4993ff700 1 Processor -- start 2023-07-06T20:15:01.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.942+0000 7fa4993ff700 1 -- start start 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.943+0000 7fa4993ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4940b5c90 0x7fa4940d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.943+0000 7fa4993ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4940b6110 0x7fa4940c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.943+0000 7fa4993ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa494152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.943+0000 7fa4993ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940c7000 con 0x7fa49415fca0 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.943+0000 7fa4993ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940c7250 con 0x7fa4940b6110 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.943+0000 7fa4993ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940c7530 con 0x7fa4940b5c90 2023-07-06T20:15:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa494152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa494152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33316/0 (socket says 172.21.15.27:33316) 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 -- 172.21.15.27:0/163661416 learned_addr learned my addr 172.21.15.27:0/163661416 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa48bfff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4940b6110 0x7fa4940c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa492ffd700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4940b5c90 0x7fa4940d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 -- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4940b5c90 msgr2=0x7fa4940d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4940b5c90 0x7fa4940d2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 -- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4940b6110 msgr2=0x7fa4940c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4940b6110 0x7fa4940c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.944+0000 7fa4937fe700 1 -- 172.21.15.27:0/163661416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484009550 con 0x7fa49415fca0 2023-07-06T20:15:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.945+0000 7fa4937fe700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa494152740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00cc50 tx=0x7fa48c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.945+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c003c00 con 0x7fa49415fca0 2023-07-06T20:15:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.945+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c003d60 con 0x7fa49415fca0 2023-07-06T20:15:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.945+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00aab0 con 0x7fa49415fca0 2023-07-06T20:15:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.945+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4940c7810 con 0x7fa49415fca0 2023-07-06T20:15:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.945+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4940c7dd0 con 0x7fa49415fca0 2023-07-06T20:15:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.946+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4940008d0 con 0x7fa49415fca0 2023-07-06T20:15:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.947+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa48c00ac10 con 0x7fa49415fca0 2023-07-06T20:15:01.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.948+0000 7fa48b7fe700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa47406ace0 0x7fa47406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.948+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa48c089410 con 0x7fa49415fca0 2023-07-06T20:15:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.948+0000 7fa492ffd700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa47406ace0 0x7fa47406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:01.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.949+0000 7fa492ffd700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa47406ace0 0x7fa47406d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa484014040 tx=0x7fa484008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:01.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:01.952+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa48c054470 con 0x7fa49415fca0 2023-07-06T20:15:02.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.211+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa49415a280 con 0x7fa49415fca0 2023-07-06T20:15:02.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.211+0000 7fa48b7fe700 1 -- 172.21.15.27:0/163661416 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7fa48c057a90 con 0x7fa49415fca0 2023-07-06T20:15:02.213 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:02.213 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:15:02.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa47406ace0 msgr2=0x7fa47406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa47406ace0 0x7fa47406d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa484014040 tx=0x7fa484008030 comp rx=0 tx=0).stop 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 msgr2=0x7fa494152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa494152740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00cc50 tx=0x7fa48c00cf60 comp rx=0 tx=0).stop 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 shutdown_connections 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa47406ace0 0x7fa47406d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa49415fca0 0x7fa494152740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4940b5c90 0x7fa4940d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 --2- 172.21.15.27:0/163661416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4940b6110 0x7fa4940c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 >> 172.21.15.27:0/163661416 conn(0x7fa49406c150 msgr2=0x7fa4940b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 shutdown_connections 2023-07-06T20:15:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.214+0000 7fa4993ff700 1 -- 172.21.15.27:0/163661416 wait complete. 2023-07-06T20:15:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:15:02.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:02.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.437+0000 7f9d99dff700 1 Processor -- start 2023-07-06T20:15:02.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.437+0000 7f9d99dff700 1 -- start start 2023-07-06T20:15:02.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d99dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d940b5c90 0x7f9d940d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d99dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d99dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d940cf780 con 0x7f9d9415fca0 2023-07-06T20:15:02.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d99dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d940cd490 con 0x7f9d940b5c90 2023-07-06T20:15:02.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d99dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940cd770 con 0x7f9d940b6110 2023-07-06T20:15:02.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d92ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d940b5c90 0x7f9d940d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d92ffd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d940b5c90 0x7f9d940d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33322/0 (socket says 172.21.15.27:33322) 2023-07-06T20:15:02.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d92ffd700 1 -- 172.21.15.27:0/1955537357 learned_addr learned my addr 172.21.15.27:0/1955537357 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:02.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.438+0000 7f9d93fff700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d92ffd700 1 -- 172.21.15.27:0/1955537357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 msgr2=0x7f9d940d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:02.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d92ffd700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d92ffd700 1 -- 172.21.15.27:0/1955537357 >> v1:172.21.15.153:6789/0 conn(0x7f9d9415fca0 legacy=0x7f9d940d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d92ffd700 1 -- 172.21.15.27:0/1955537357 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940cda50 con 0x7f9d940b5c90 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d93fff700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d92ffd700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d940b5c90 0x7f9d940d2d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098f0 tx=0x7f9d88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d4aac02eda438d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d927fc700 1 -- 172.21.15.27:0/1955537357 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800b8a0 con 0x7f9d940b5c90 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d927fc700 1 -- 172.21.15.27:0/1955537357 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8800ba00 con 0x7f9d940b5c90 2023-07-06T20:15:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.439+0000 7f9d927fc700 1 -- 172.21.15.27:0/1955537357 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800bd90 con 0x7f9d940b5c90 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 -- 172.21.15.27:0/1955537357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d940b5c90 msgr2=0x7f9d940d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d940b5c90 0x7f9d940d2d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098f0 tx=0x7f9d88009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 -- 172.21.15.27:0/1955537357 shutdown_connections 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 --2- 172.21.15.27:0/1955537357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d940b5c90 0x7f9d940d2d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 -- 172.21.15.27:0/1955537357 >> 172.21.15.27:0/1955537357 conn(0x7f9d9406c150 msgr2=0x7f9d9406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 -- 172.21.15.27:0/1955537357 shutdown_connections 2023-07-06T20:15:02.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.440+0000 7f9d99dff700 1 -- 172.21.15.27:0/1955537357 wait complete. 2023-07-06T20:15:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.442+0000 7f9d99dff700 1 Processor -- start 2023-07-06T20:15:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.442+0000 7f9d99dff700 1 -- start start 2023-07-06T20:15:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d99dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d940b5c90 0x7f9d94157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d99dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d99dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 0x7f9d940d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d99dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94157bd0 con 0x7f9d9415fca0 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d99dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c8720 con 0x7f9d940b5c90 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d99dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c8a00 con 0x7f9d940b6110 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 0x7f9d940d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 0x7f9d940d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33338/0 (socket says 172.21.15.27:33338) 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 -- 172.21.15.27:0/3030467640 learned_addr learned my addr 172.21.15.27:0/3030467640 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d937fe700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d940b5c90 0x7f9d94157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d92ffd700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 -- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 msgr2=0x7f9d940c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 -- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d940b5c90 msgr2=0x7f9d94157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d940b5c90 0x7f9d94157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d93fff700 1 -- 172.21.15.27:0/3030467640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009550 con 0x7f9d9415fca0 2023-07-06T20:15:02.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.443+0000 7f9d937fe700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d940b5c90 0x7f9d94157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:02.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.444+0000 7f9d93fff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 0x7f9d940d0960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007ef0 tx=0x7f9d8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:02.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.444+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00de60 con 0x7f9d9415fca0 2023-07-06T20:15:02.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.444+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c00f040 con 0x7f9d9415fca0 2023-07-06T20:15:02.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.444+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c003c60 con 0x7f9d9415fca0 2023-07-06T20:15:02.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.444+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d940cb070 con 0x7f9d9415fca0 2023-07-06T20:15:02.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.444+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d940c8ce0 con 0x7f9d9415fca0 2023-07-06T20:15:02.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.445+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d940cb590 con 0x7f9d9415fca0 2023-07-06T20:15:02.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.446+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d8c004030 con 0x7f9d9415fca0 2023-07-06T20:15:02.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.447+0000 7f9d7bfff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9d8006ace0 0x7f9d8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.447+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9d8c089630 con 0x7f9d9415fca0 2023-07-06T20:15:02.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.448+0000 7f9d937fe700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9d8006ace0 0x7f9d8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.448+0000 7f9d937fe700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9d8006ace0 0x7f9d8006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000f60 tx=0x7f9d84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:02.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.452+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d940cb590 con 0x7f9d9415fca0 2023-07-06T20:15:02.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.658+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d940cb590 con 0x7f9d9415fca0 2023-07-06T20:15:02.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.659+0000 7f9d7bfff700 1 -- 172.21.15.27:0/3030467640 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f9d940cb590 con 0x7f9d9415fca0 2023-07-06T20:15:02.661 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:02.662 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:02.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9d8006ace0 msgr2=0x7f9d8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9d8006ace0 0x7f9d8006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000f60 tx=0x7f9d84009040 comp rx=0 tx=0).stop 2023-07-06T20:15:02.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 msgr2=0x7f9d940d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 0x7f9d940d0960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007ef0 tx=0x7f9d8c00d3b0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 shutdown_connections 2023-07-06T20:15:02.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9d8006ace0 0x7f9d8006d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d9415fca0 0x7f9d940d0960 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d940b5c90 0x7f9d94157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 --2- 172.21.15.27:0/3030467640 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9d940b6110 0x7f9d940c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 >> 172.21.15.27:0/3030467640 conn(0x7f9d9406c150 msgr2=0x7f9d940b8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:02.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 shutdown_connections 2023-07-06T20:15:02.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.662+0000 7f9d99dff700 1 -- 172.21.15.27:0/3030467640 wait complete. 2023-07-06T20:15:02.682 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T20:15:02.682 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:02.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.885+0000 7fad197ff700 1 Processor -- start 2023-07-06T20:15:02.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.885+0000 7fad197ff700 1 -- start start 2023-07-06T20:15:02.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.886+0000 7fad197ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad140b6110 0x7fad140cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.886+0000 7fad197ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad140cf780 con 0x7fad140b5c90 2023-07-06T20:15:02.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.886+0000 7fad197ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad140cd490 con 0x7fad1415fca0 2023-07-06T20:15:02.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.886+0000 7fad197ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad140cd770 con 0x7fad140b6110 2023-07-06T20:15:02.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad127fc700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fad140b5c90 0x7fad140d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53026/0 (socket says 172.21.15.27:53026) 2023-07-06T20:15:02.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad127fc700 1 -- 172.21.15.27:0/3828723828 learned_addr learned my addr 172.21.15.27:0/3828723828 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:02.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad137fe700 1 --2- 172.21.15.27:0/3828723828 >> v2:172.21.15.27:3300/0 conn(0x7fad140b6110 0x7fad140cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad11ffb700 1 -- 172.21.15.27:0/3828723828 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176098793 0 0) 0x7fad140cf780 con 0x7fad140b5c90 2023-07-06T20:15:02.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad11ffb700 1 -- 172.21.15.27:0/3828723828 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facfc0035b0 con 0x7fad140b5c90 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad137fe700 1 -- 172.21.15.27:0/3828723828 >> v1:172.21.15.153:6789/0 conn(0x7fad1415fca0 legacy=0x7fad140d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad11ffb700 1 -- 172.21.15.27:0/3828723828 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697932978 0 0) 0x7fad140cd490 con 0x7fad1415fca0 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad137fe700 1 -- 172.21.15.27:0/3828723828 >> v1:172.21.15.27:6789/0 conn(0x7fad140b5c90 legacy=0x7fad140d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.887+0000 7fad137fe700 1 -- 172.21.15.27:0/3828723828 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad140cda50 con 0x7fad140b6110 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad137fe700 1 --2- 172.21.15.27:0/3828723828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad140b6110 0x7fad140cfc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009d90 tx=0x7fad0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53bb3f4105b3b55 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad11ffb700 1 -- 172.21.15.27:0/3828723828 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00e070 con 0x7fad140b6110 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad11ffb700 1 -- 172.21.15.27:0/3828723828 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c00bb20 con 0x7fad140b6110 2023-07-06T20:15:02.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad11ffb700 1 -- 172.21.15.27:0/3828723828 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00bed0 con 0x7fad140b6110 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad197ff700 1 -- 172.21.15.27:0/3828723828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad140b6110 msgr2=0x7fad140cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad197ff700 1 --2- 172.21.15.27:0/3828723828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad140b6110 0x7fad140cfc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009d90 tx=0x7fad0c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad197ff700 1 -- 172.21.15.27:0/3828723828 shutdown_connections 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad197ff700 1 --2- 172.21.15.27:0/3828723828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad140b6110 0x7fad140cfc10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.888+0000 7fad197ff700 1 -- 172.21.15.27:0/3828723828 >> 172.21.15.27:0/3828723828 conn(0x7fad1406c150 msgr2=0x7fad1406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.889+0000 7fad197ff700 1 -- 172.21.15.27:0/3828723828 shutdown_connections 2023-07-06T20:15:02.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.889+0000 7fad197ff700 1 -- 172.21.15.27:0/3828723828 wait complete. 2023-07-06T20:15:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.890+0000 7fad197ff700 1 Processor -- start 2023-07-06T20:15:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.890+0000 7fad197ff700 1 -- start start 2023-07-06T20:15:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.891+0000 7fad197ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fad140b5c90 0x7fad141f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.891+0000 7fad197ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad140b6110 0x7fad140c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.891+0000 7fad197ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 0x7fad14156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.891+0000 7fad197ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad140c6e70 con 0x7fad1415fca0 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.891+0000 7fad197ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad140c7300 con 0x7fad140b6110 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.891+0000 7fad197ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad140c7550 con 0x7fad140b5c90 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 0x7fad14156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 0x7fad14156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33352/0 (socket says 172.21.15.27:33352) 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 -- 172.21.15.27:0/826620181 learned_addr learned my addr 172.21.15.27:0/826620181 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad12ffd700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fad140b5c90 0x7fad141f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad127fc700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad140b6110 0x7fad140c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 -- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fad140b5c90 msgr2=0x7fad141f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fad140b5c90 0x7fad141f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 -- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad140b6110 msgr2=0x7fad140c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad140b6110 0x7fad140c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 -- 172.21.15.27:0/826620181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c009a70 con 0x7fad1415fca0 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad12ffd700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fad140b5c90 0x7fad141f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad127fc700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad140b6110 0x7fad140c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad137fe700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 0x7fad14156f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fad0c014040 tx=0x7fad0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c01b810 con 0x7fad1415fca0 2023-07-06T20:15:02.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c020070 con 0x7fad1415fca0 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.892+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad140c7830 con 0x7fad1415fca0 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.893+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c01bbb0 con 0x7fad1415fca0 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.893+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad140c7df0 con 0x7fad1415fca0 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.894+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad140008d0 con 0x7fad1415fca0 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.895+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad0c01bd10 con 0x7fad1415fca0 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.896+0000 7facfb7fe700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fad0006ace0 0x7fad0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.896+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fad0c022070 con 0x7fad1415fca0 2023-07-06T20:15:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.896+0000 7fad12ffd700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fad0006ace0 0x7fad0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.897+0000 7fad12ffd700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fad0006ace0 0x7fad0006d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad04000f60 tx=0x7fad04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:02.900+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fad0c05a4b0 con 0x7fad1415fca0 2023-07-06T20:15:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.156+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad140ccd20 con 0x7fad1415fca0 2023-07-06T20:15:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.156+0000 7facfb7fe700 1 -- 172.21.15.27:0/826620181 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7fad0c05a040 con 0x7fad1415fca0 2023-07-06T20:15:03.158 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:03.158 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:15:03.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.158+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fad0006ace0 msgr2=0x7fad0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.158+0000 7fad197ff700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fad0006ace0 0x7fad0006d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad04000f60 tx=0x7fad04009040 comp rx=0 tx=0).stop 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.158+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 msgr2=0x7fad14156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.158+0000 7fad197ff700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 0x7fad14156f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fad0c014040 tx=0x7fad0c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 shutdown_connections 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fad0006ace0 0x7fad0006d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad1415fca0 0x7fad14156f60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fad140b5c90 0x7fad141f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 --2- 172.21.15.27:0/826620181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fad140b6110 0x7fad140c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 >> 172.21.15.27:0/826620181 conn(0x7fad1406c150 msgr2=0x7fad140b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 shutdown_connections 2023-07-06T20:15:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.159+0000 7fad197ff700 1 -- 172.21.15.27:0/826620181 wait complete. 2023-07-06T20:15:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:15:03.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:03.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.380+0000 7f5c8a1ff700 1 Processor -- start 2023-07-06T20:15:03.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.380+0000 7f5c8a1ff700 1 -- start start 2023-07-06T20:15:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.381+0000 7f5c8a1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5c840b6110 0x7f5c840cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.381+0000 7f5c8a1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c840cf780 con 0x7f5c8415fca0 2023-07-06T20:15:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.381+0000 7f5c8a1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c840cd490 con 0x7f5c840b5c90 2023-07-06T20:15:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.381+0000 7f5c8a1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c840cd770 con 0x7f5c840b6110 2023-07-06T20:15:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c837fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5c8415fca0 0x7f5c840d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53040/0 (socket says 172.21.15.27:53040) 2023-07-06T20:15:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c837fe700 1 -- 172.21.15.27:0/1330962211 learned_addr learned my addr 172.21.15.27:0/1330962211 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c83fff700 1 --2- 172.21.15.27:0/1330962211 >> v2:172.21.15.153:3300/0 conn(0x7f5c840b6110 0x7f5c840cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c827fc700 1 -- 172.21.15.27:0/1330962211 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652220511 0 0) 0x7f5c840cf780 con 0x7f5c8415fca0 2023-07-06T20:15:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c827fc700 1 -- 172.21.15.27:0/1330962211 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c6c0035b0 con 0x7f5c8415fca0 2023-07-06T20:15:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c83fff700 1 -- 172.21.15.27:0/1330962211 >> v1:172.21.15.153:6789/0 conn(0x7f5c840b5c90 legacy=0x7f5c840d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c83fff700 1 -- 172.21.15.27:0/1330962211 >> v1:172.21.15.27:6789/0 conn(0x7f5c8415fca0 legacy=0x7f5c840d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c827fc700 1 -- 172.21.15.27:0/1330962211 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3366500122 0 0) 0x7f5c6c0035b0 con 0x7f5c8415fca0 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.382+0000 7f5c83fff700 1 -- 172.21.15.27:0/1330962211 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840cda50 con 0x7f5c840b6110 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.383+0000 7f5c83fff700 1 --2- 172.21.15.27:0/1330962211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c840cfc10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009d90 tx=0x7f5c7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22bae7adaec2481 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.383+0000 7f5c827fc700 1 -- 172.21.15.27:0/1330962211 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00e070 con 0x7f5c840b6110 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.383+0000 7f5c827fc700 1 -- 172.21.15.27:0/1330962211 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c003a20 con 0x7f5c840b6110 2023-07-06T20:15:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.383+0000 7f5c827fc700 1 -- 172.21.15.27:0/1330962211 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74001670 con 0x7f5c840b6110 2023-07-06T20:15:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/1330962211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 msgr2=0x7f5c840cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/1330962211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c840cfc10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009d90 tx=0x7f5c7c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/1330962211 shutdown_connections 2023-07-06T20:15:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/1330962211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c840cfc10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/1330962211 >> 172.21.15.27:0/1330962211 conn(0x7f5c8406c150 msgr2=0x7f5c8406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/1330962211 shutdown_connections 2023-07-06T20:15:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.384+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/1330962211 wait complete. 2023-07-06T20:15:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.386+0000 7f5c8a1ff700 1 Processor -- start 2023-07-06T20:15:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.386+0000 7f5c8a1ff700 1 -- start start 2023-07-06T20:15:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c8a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c840b5c90 0x7f5c840cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c8a1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c8415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c8a1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 0x7f5c84160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c8a1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8415cf90 con 0x7f5c8415fca0 2023-07-06T20:15:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c8a1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8415c380 con 0x7f5c840b6110 2023-07-06T20:15:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c8a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8415c660 con 0x7f5c840b5c90 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 0x7f5c84160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 0x7f5c84160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33356/0 (socket says 172.21.15.27:33356) 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 -- 172.21.15.27:0/4209948975 learned_addr learned my addr 172.21.15.27:0/4209948975 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c837fe700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c840b5c90 0x7f5c840cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c82ffd700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c8415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 -- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c840b5c90 msgr2=0x7f5c840cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c840b5c90 0x7f5c840cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 -- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 msgr2=0x7f5c8415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c8415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.387+0000 7f5c83fff700 1 -- 172.21.15.27:0/4209948975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c009a70 con 0x7f5c8415fca0 2023-07-06T20:15:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c82ffd700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c8415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c83fff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 0x7f5c84160070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5c8406d0f0 tx=0x7f5c7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:03.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004760 con 0x7f5c8415fca0 2023-07-06T20:15:03.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0048e0 con 0x7f5c8415fca0 2023-07-06T20:15:03.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c012670 con 0x7f5c8415fca0 2023-07-06T20:15:03.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8415c940 con 0x7f5c8415fca0 2023-07-06T20:15:03.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.388+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c840cca70 con 0x7f5c8415fca0 2023-07-06T20:15:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.389+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c840ce950 con 0x7f5c8415fca0 2023-07-06T20:15:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.390+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c7c0127d0 con 0x7f5c8415fca0 2023-07-06T20:15:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.391+0000 7f5c6bfff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5c7006ace0 0x7f5c7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.391+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5c7c08af50 con 0x7f5c8415fca0 2023-07-06T20:15:03.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.392+0000 7f5c837fe700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5c7006ace0 0x7f5c7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.392+0000 7f5c837fe700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5c7006ace0 0x7f5c7006d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c740011e0 tx=0x7f5c74007a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:03.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.396+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c7c059b60 con 0x7f5c8415fca0 2023-07-06T20:15:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.649+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c840008d0 con 0x7f5c8415fca0 2023-07-06T20:15:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.650+0000 7f5c6bfff700 1 -- 172.21.15.27:0/4209948975 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f5c7c0596f0 con 0x7f5c8415fca0 2023-07-06T20:15:03.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2038912547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2038912547},{"type":"v1","addr":"172.21.15.27:6835","nonce":2038912547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/408963482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":408963482},{"type":"v1","addr":"172.21.15.153:6837","nonce":408963482}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/3315267874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":3315267874},{"type":"v1","addr":"172.21.15.153:6836","nonce":3315267874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:13:59.519602+0000","modified":"2023-07-06T20:14:03.050155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3572656483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3572656483},{"type":"v1","addr":"172.21.15.27:6834","nonce":3572656483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:15:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5c7006ace0 msgr2=0x7f5c7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5c7006ace0 0x7f5c7006d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c740011e0 tx=0x7f5c74007a20 comp rx=0 tx=0).stop 2023-07-06T20:15:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 msgr2=0x7f5c84160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 0x7f5c84160070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5c8406d0f0 tx=0x7f5c7c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 shutdown_connections 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5c7006ace0 0x7f5c7006d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8415fca0 0x7f5c84160070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5c840b5c90 0x7f5c840cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 --2- 172.21.15.27:0/4209948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c840b6110 0x7f5c8415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 >> 172.21.15.27:0/4209948975 conn(0x7f5c8406c150 msgr2=0x7f5c840b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:03.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 shutdown_connections 2023-07-06T20:15:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.652+0000 7f5c8a1ff700 1 -- 172.21.15.27:0/4209948975 wait complete. 2023-07-06T20:15:03.656 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2023-07-06T20:15:03.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:03.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.874+0000 7fdf8fdff700 1 Processor -- start 2023-07-06T20:15:03.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.874+0000 7fdf8fdff700 1 -- start start 2023-07-06T20:15:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8fdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf8815fca0 0x7fdf880d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8fdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf880b5c90 0x7fdf880d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8fdff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdf880b6100 0x7fdf880d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8fdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf880cf780 con 0x7fdf880b5c90 2023-07-06T20:15:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8fdff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf880cd490 con 0x7fdf880b6100 2023-07-06T20:15:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8fdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880cd770 con 0x7fdf8815fca0 2023-07-06T20:15:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8d39a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf880b5c90 0x7fdf880d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8d39a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf880b5c90 0x7fdf880d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33358/0 (socket says 172.21.15.27:33358) 2023-07-06T20:15:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8d39a700 1 -- 172.21.15.27:0/4008819437 learned_addr learned my addr 172.21.15.27:0/4008819437 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:03.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8db9b700 1 --2- 172.21.15.27:0/4008819437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf8815fca0 0x7fdf880d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.875+0000 7fdf8e39c700 1 --2- 172.21.15.27:0/4008819437 >> v2:172.21.15.153:3300/0 conn(0x7fdf880b6100 0x7fdf880d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8d39a700 1 -- 172.21.15.27:0/4008819437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf8815fca0 msgr2=0x7fdf880d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:03.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8d39a700 1 --2- 172.21.15.27:0/4008819437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf8815fca0 0x7fdf880d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8d39a700 1 -- 172.21.15.27:0/4008819437 >> v2:172.21.15.153:3300/0 conn(0x7fdf880b6100 msgr2=0x7fdf880d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:03.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8d39a700 1 --2- 172.21.15.27:0/4008819437 >> v2:172.21.15.153:3300/0 conn(0x7fdf880b6100 0x7fdf880d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8d39a700 1 -- 172.21.15.27:0/4008819437 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880cda50 con 0x7fdf880b5c90 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8d39a700 1 --2- 172.21.15.27:0/4008819437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf880b5c90 0x7fdf880d2d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdf880b8f80 tx=0x7fdf78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345ebd751e565bd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8cb99700 1 -- 172.21.15.27:0/4008819437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800b870 con 0x7fdf880b5c90 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8cb99700 1 -- 172.21.15.27:0/4008819437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800b9d0 con 0x7fdf880b5c90 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.876+0000 7fdf8cb99700 1 -- 172.21.15.27:0/4008819437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800bd60 con 0x7fdf880b5c90 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 -- 172.21.15.27:0/4008819437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf880b5c90 msgr2=0x7fdf880d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/4008819437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf880b5c90 0x7fdf880d2d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdf880b8f80 tx=0x7fdf78009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 -- 172.21.15.27:0/4008819437 shutdown_connections 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/4008819437 >> v2:172.21.15.153:3300/0 conn(0x7fdf880b6100 0x7fdf880d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/4008819437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf8815fca0 0x7fdf880d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/4008819437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf880b5c90 0x7fdf880d2d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 -- 172.21.15.27:0/4008819437 >> 172.21.15.27:0/4008819437 conn(0x7fdf8806c150 msgr2=0x7fdf8806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 -- 172.21.15.27:0/4008819437 shutdown_connections 2023-07-06T20:15:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.877+0000 7fdf8fdff700 1 -- 172.21.15.27:0/4008819437 wait complete. 2023-07-06T20:15:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.879+0000 7fdf8fdff700 1 Processor -- start 2023-07-06T20:15:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.879+0000 7fdf8fdff700 1 -- start start 2023-07-06T20:15:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8fdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf880b5c90 0x7fdf8815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8fdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf880b6100 0x7fdf880aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8fdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 0x7fdf8814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8fdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8815d530 con 0x7fdf8815fca0 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8fdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8815d810 con 0x7fdf880b5c90 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8fdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8815daf0 con 0x7fdf880b6100 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8e39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 0x7fdf8814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8e39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 0x7fdf8814f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33364/0 (socket says 172.21.15.27:33364) 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8e39c700 1 -- 172.21.15.27:0/1584031609 learned_addr learned my addr 172.21.15.27:0/1584031609 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8db9b700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf880b5c90 0x7fdf8815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8d39a700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf880b6100 0x7fdf880aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.880+0000 7fdf8e39c700 1 -- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf880b6100 msgr2=0x7fdf880aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8e39c700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf880b6100 0x7fdf880aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8e39c700 1 -- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf880b5c90 msgr2=0x7fdf8815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8e39c700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf880b5c90 0x7fdf8815e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8e39c700 1 -- 172.21.15.27:0/1584031609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009550 con 0x7fdf8815fca0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8e39c700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 0x7fdf8814f280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdf80007d10 tx=0x7fdf8000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000ed30 con 0x7fdf8815fca0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8000ee90 con 0x7fdf8815fca0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8815ddd0 con 0x7fdf8815fca0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80014660 con 0x7fdf8815fca0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.881+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf880aa620 con 0x7fdf8815fca0 2023-07-06T20:15:03.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.882+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8815e2f0 con 0x7fdf8815fca0 2023-07-06T20:15:03.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.883+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf800147c0 con 0x7fdf8815fca0 2023-07-06T20:15:03.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.884+0000 7fdf7e7fc700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:03.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.884+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdf80089ac0 con 0x7fdf8815fca0 2023-07-06T20:15:03.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.885+0000 7fdf8db9b700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:03.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.886+0000 7fdf8db9b700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdf6c06ace0 0x7fdf6c06d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006b20 tx=0x7fdf84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:03.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:03.889+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf8815e2f0 con 0x7fdf8815fca0 2023-07-06T20:15:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.100+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf8815e2f0 con 0x7fdf8815fca0 2023-07-06T20:15:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.101+0000 7fdf7e7fc700 1 -- 172.21.15.27:0/1584031609 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18350 (secure 0 0 0) 0x7fdf8815e2f0 con 0x7fdf8815fca0 2023-07-06T20:15:04.103 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:04.103 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:14:39.964072+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:13:57.143873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T20:13:58.519309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.103+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdf6c06ace0 msgr2=0x7fdf6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.103+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdf6c06ace0 0x7fdf6c06d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006b20 tx=0x7fdf84008040 comp rx=0 tx=0).stop 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.103+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 msgr2=0x7fdf8814f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.103+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 0x7fdf8814f280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdf80007d10 tx=0x7fdf8000e3c0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.103+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 shutdown_connections 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf8815fca0 0x7fdf8814f280 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf880b5c90 0x7fdf8815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 --2- 172.21.15.27:0/1584031609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf880b6100 0x7fdf880aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 >> 172.21.15.27:0/1584031609 conn(0x7fdf8806c150 msgr2=0x7fdf880cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 shutdown_connections 2023-07-06T20:15:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.104+0000 7fdf8fdff700 1 -- 172.21.15.27:0/1584031609 wait complete. 2023-07-06T20:15:04.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T20:15:04.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.327+0000 7f35e51ff700 1 Processor -- start 2023-07-06T20:15:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.327+0000 7f35e51ff700 1 -- start start 2023-07-06T20:15:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.328+0000 7f35e51ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b5c90 0x7f35e00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.328+0000 7f35e51ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f35e00b60e0 0x7f35e00d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.328+0000 7f35e51ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e00cf780 con 0x7f35e015fca0 2023-07-06T20:15:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.328+0000 7f35e51ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e00cd490 con 0x7f35e00b60e0 2023-07-06T20:15:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.328+0000 7f35e51ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00cd770 con 0x7f35e00b5c90 2023-07-06T20:15:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.328+0000 7f35ded9d700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f35e015fca0 0x7f35e00d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53044/0 (socket says 172.21.15.27:53044) 2023-07-06T20:15:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35ded9d700 1 -- 172.21.15.27:0/1621478936 learned_addr learned my addr 172.21.15.27:0/1621478936 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35de59c700 1 --2- 172.21.15.27:0/1621478936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b5c90 0x7f35e00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35df59e700 1 --2- 172.21.15.27:0/1621478936 >> v2:172.21.15.153:3300/0 conn(0x7f35e00b60e0 0x7f35e00d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35ddd9b700 1 -- 172.21.15.27:0/1621478936 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363757836 0 0) 0x7f35e00cf780 con 0x7f35e015fca0 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35ddd9b700 1 -- 172.21.15.27:0/1621478936 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c80035b0 con 0x7f35e015fca0 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35de59c700 1 -- 172.21.15.27:0/1621478936 >> v2:172.21.15.153:3300/0 conn(0x7f35e00b60e0 msgr2=0x7f35e00d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35de59c700 1 --2- 172.21.15.27:0/1621478936 >> v2:172.21.15.153:3300/0 conn(0x7f35e00b60e0 0x7f35e00d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35de59c700 1 -- 172.21.15.27:0/1621478936 >> v1:172.21.15.27:6789/0 conn(0x7f35e015fca0 legacy=0x7f35e00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.329+0000 7f35de59c700 1 -- 172.21.15.27:0/1621478936 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e00cda50 con 0x7f35e00b5c90 2023-07-06T20:15:04.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.330+0000 7f35de59c700 1 --2- 172.21.15.27:0/1621478936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b5c90 0x7f35e00d2d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009840 tx=0x7f35d4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12241b09b8464327 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.330+0000 7f35ddd9b700 1 -- 172.21.15.27:0/1621478936 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011830 con 0x7f35e00b5c90 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.330+0000 7f35ddd9b700 1 -- 172.21.15.27:0/1621478936 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d4011990 con 0x7f35e00b5c90 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.330+0000 7f35ddd9b700 1 -- 172.21.15.27:0/1621478936 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011d20 con 0x7f35e00b5c90 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.330+0000 7f35e51ff700 1 -- 172.21.15.27:0/1621478936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b5c90 msgr2=0x7f35e00d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.330+0000 7f35e51ff700 1 --2- 172.21.15.27:0/1621478936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b5c90 0x7f35e00d2d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009840 tx=0x7f35d4009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.331+0000 7f35e51ff700 1 -- 172.21.15.27:0/1621478936 shutdown_connections 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.331+0000 7f35e51ff700 1 --2- 172.21.15.27:0/1621478936 >> v2:172.21.15.153:3300/0 conn(0x7f35e00b60e0 0x7f35e00d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.331+0000 7f35e51ff700 1 --2- 172.21.15.27:0/1621478936 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b5c90 0x7f35e00d2d40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.331+0000 7f35e51ff700 1 -- 172.21.15.27:0/1621478936 >> 172.21.15.27:0/1621478936 conn(0x7f35e006c150 msgr2=0x7f35e006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.331+0000 7f35e51ff700 1 -- 172.21.15.27:0/1621478936 shutdown_connections 2023-07-06T20:15:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.331+0000 7f35e51ff700 1 -- 172.21.15.27:0/1621478936 wait complete. 2023-07-06T20:15:04.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.332+0000 7f35e51ff700 1 Processor -- start 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.333+0000 7f35e51ff700 1 -- start start 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35e51ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e00b5c90 0x7f35e0157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35e51ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b60e0 0x7f35e00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35e51ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 0x7f35e0159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35e51ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e015b990 con 0x7f35e015fca0 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35e51ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e015bc70 con 0x7f35e00b5c90 2023-07-06T20:15:04.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35e51ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e015bf50 con 0x7f35e00b60e0 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 0x7f35e0159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 0x7f35e0159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33376/0 (socket says 172.21.15.27:33376) 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35ded9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e00b5c90 0x7f35e0157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 -- 172.21.15.27:0/2994244899 learned_addr learned my addr 172.21.15.27:0/2994244899 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35de59c700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b60e0 0x7f35e00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 -- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b60e0 msgr2=0x7f35e00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b60e0 0x7f35e00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 -- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e00b5c90 msgr2=0x7f35e0157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e00b5c90 0x7f35e0157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.334+0000 7f35df59e700 1 -- 172.21.15.27:0/2994244899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40094a0 con 0x7f35e015fca0 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.335+0000 7f35df59e700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 0x7f35e0159410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f35d800efc0 tx=0x7f35d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.335+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800cc90 con 0x7f35e015fca0 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.335+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d800f040 con 0x7f35e015fca0 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.335+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e015c230 con 0x7f35e015fca0 2023-07-06T20:15:04.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.335+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d80146c0 con 0x7f35e015fca0 2023-07-06T20:15:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.335+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e00b37c0 con 0x7f35e015fca0 2023-07-06T20:15:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.336+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e015c750 con 0x7f35e015fca0 2023-07-06T20:15:04.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.337+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35d8014820 con 0x7f35e015fca0 2023-07-06T20:15:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.338+0000 7f35c77fe700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f35cc06ace0 0x7f35cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.338+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f35d808a000 con 0x7f35e015fca0 2023-07-06T20:15:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.339+0000 7f35ded9d700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f35cc06ace0 0x7f35cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.339+0000 7f35ded9d700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f35cc06ace0 0x7f35cc06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35d0007fd0 tx=0x7f35d0007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:04.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.342+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35e015c750 con 0x7f35e015fca0 2023-07-06T20:15:04.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.599+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35e015c750 con 0x7f35e015fca0 2023-07-06T20:15:04.601 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-07-06T20:15:04.600+0000 7f859e7f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T20:15:04.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.604+0000 7f35c77fe700 1 -- 172.21.15.27:0/2994244899 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f35e015c750 con 0x7f35e015fca0 2023-07-06T20:15:04.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.607+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f35cc06ace0 msgr2=0x7f35cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.607+0000 7f35e51ff700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f35cc06ace0 0x7f35cc06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35d0007fd0 tx=0x7f35d0007dc0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.607+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 msgr2=0x7f35e0159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.607+0000 7f35e51ff700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 0x7f35e0159410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f35d800efc0 tx=0x7f35d800c3a0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 shutdown_connections 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f35cc06ace0 0x7f35cc06d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35e015fca0 0x7f35e0159410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e00b5c90 0x7f35e0157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 --2- 172.21.15.27:0/2994244899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35e00b60e0 0x7f35e00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 >> 172.21.15.27:0/2994244899 conn(0x7f35e006c150 msgr2=0x7f35e00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:04.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 shutdown_connections 2023-07-06T20:15:04.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.608+0000 7f35e51ff700 1 -- 172.21.15.27:0/2994244899 wait complete. 2023-07-06T20:15:04.611 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T20:15:04.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T20:15:04.679 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-07-06T20:15:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.840+0000 7f357e7ff700 1 Processor -- start 2023-07-06T20:15:04.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.841+0000 7f357e7ff700 1 -- start start 2023-07-06T20:15:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f357e7ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f357815fca0 0x7f35780d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f357e7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35780b5c90 0x7f35780d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f357e7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f357e7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35780cf780 con 0x7f35780b5c90 2023-07-06T20:15:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f357e7ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35780cd490 con 0x7f357815fca0 2023-07-06T20:15:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f357e7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780cd770 con 0x7f35780b6110 2023-07-06T20:15:04.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f356ffff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35780b5c90 0x7f35780d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f356ffff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35780b5c90 0x7f35780d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33392/0 (socket says 172.21.15.27:33392) 2023-07-06T20:15:04.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f356ffff700 1 -- 172.21.15.27:0/2285832501 learned_addr learned my addr 172.21.15.27:0/2285832501 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:04.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.842+0000 7f3577fff700 1 --2- 172.21.15.27:0/2285832501 >> v2:172.21.15.153:3300/0 conn(0x7f357815fca0 0x7f35780d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f357cd9c700 1 --2- 172.21.15.27:0/2285832501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f356ffff700 1 -- 172.21.15.27:0/2285832501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 msgr2=0x7f35780d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f356ffff700 1 --2- 172.21.15.27:0/2285832501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f356ffff700 1 -- 172.21.15.27:0/2285832501 >> v2:172.21.15.153:3300/0 conn(0x7f357815fca0 msgr2=0x7f35780d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f356ffff700 1 --2- 172.21.15.27:0/2285832501 >> v2:172.21.15.153:3300/0 conn(0x7f357815fca0 0x7f35780d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f356ffff700 1 -- 172.21.15.27:0/2285832501 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780cda50 con 0x7f35780b5c90 2023-07-06T20:15:04.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f3577fff700 1 --2- 172.21.15.27:0/2285832501 >> v2:172.21.15.153:3300/0 conn(0x7f357815fca0 0x7f35780d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:04.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f356ffff700 1 --2- 172.21.15.27:0/2285832501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35780b5c90 0x7f35780d2d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f35780d0930 tx=0x7f3560009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e32e53006139a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f35777fe700 1 -- 172.21.15.27:0/2285832501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000b870 con 0x7f35780b5c90 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f35777fe700 1 -- 172.21.15.27:0/2285832501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356000b9d0 con 0x7f35780b5c90 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.843+0000 7f35777fe700 1 -- 172.21.15.27:0/2285832501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000bd60 con 0x7f35780b5c90 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 -- 172.21.15.27:0/2285832501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35780b5c90 msgr2=0x7f35780d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 --2- 172.21.15.27:0/2285832501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35780b5c90 0x7f35780d2d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f35780d0930 tx=0x7f3560009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 -- 172.21.15.27:0/2285832501 shutdown_connections 2023-07-06T20:15:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 --2- 172.21.15.27:0/2285832501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 --2- 172.21.15.27:0/2285832501 >> v2:172.21.15.153:3300/0 conn(0x7f357815fca0 0x7f35780d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 --2- 172.21.15.27:0/2285832501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35780b5c90 0x7f35780d2d40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 -- 172.21.15.27:0/2285832501 >> 172.21.15.27:0/2285832501 conn(0x7f357806c150 msgr2=0x7f357806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 -- 172.21.15.27:0/2285832501 shutdown_connections 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.844+0000 7f357e7ff700 1 -- 172.21.15.27:0/2285832501 wait complete. 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.846+0000 7f357e7ff700 1 Processor -- start 2023-07-06T20:15:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.846+0000 7f357e7ff700 1 -- start start 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357e7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35780b5c90 0x7f3578157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357e7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357e7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 0x7f35780b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357e7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35781f2220 con 0x7f357815fca0 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357e7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357815c480 con 0x7f35780b5c90 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357e7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357815c760 con 0x7f35780b6110 2023-07-06T20:15:04.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.847+0000 7f357cd9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 0x7f35780b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 0x7f35780b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33394/0 (socket says 172.21.15.27:33394) 2023-07-06T20:15:04.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 -- 172.21.15.27:0/145822268 learned_addr learned my addr 172.21.15.27:0/145822268 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:04.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f356ffff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f3577fff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35780b5c90 0x7f3578157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 -- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 msgr2=0x7f35780b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 -- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35780b5c90 msgr2=0x7f3578157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:04.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35780b5c90 0x7f3578157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:04.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 -- 172.21.15.27:0/145822268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3560009550 con 0x7f357815fca0 2023-07-06T20:15:04.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f357cd9c700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 0x7f35780b3e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3570007e60 tx=0x7f357000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:04.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357000ede0 con 0x7f357815fca0 2023-07-06T20:15:04.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3570014420 con 0x7f357815fca0 2023-07-06T20:15:04.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.848+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35700147d0 con 0x7f357815fca0 2023-07-06T20:15:04.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.849+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357815ca40 con 0x7f357815fca0 2023-07-06T20:15:04.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.849+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3578157860 con 0x7f357815fca0 2023-07-06T20:15:04.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.850+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35781f1ff0 con 0x7f357815fca0 2023-07-06T20:15:04.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.851+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3570014930 con 0x7f357815fca0 2023-07-06T20:15:04.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.852+0000 7f35757fa700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f355806ad10 0x7f355806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:04.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.852+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f3570089c40 con 0x7f357815fca0 2023-07-06T20:15:04.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.852+0000 7f3577fff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f355806ad10 0x7f355806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:04.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.853+0000 7f3577fff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f355806ad10 0x7f355806d1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3568009930 tx=0x7f3568008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:04.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:04.856+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35781f1ff0 con 0x7f357815fca0 2023-07-06T20:15:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.117+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f35781f1ff0 con 0x7f357815fca0 2023-07-06T20:15:05.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.606+0000 7f35757fa700 1 -- 172.21.15.27:0/145822268 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f35781f1ff0 con 0x7f357815fca0 2023-07-06T20:15:05.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f355806ad10 msgr2=0x7f355806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:05.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f355806ad10 0x7f355806d1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3568009930 tx=0x7f3568008040 comp rx=0 tx=0).stop 2023-07-06T20:15:05.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 msgr2=0x7f35780b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:05.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 0x7f35780b3e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3570007e60 tx=0x7f357000e3c0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 shutdown_connections 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f355806ad10 0x7f355806d1c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f357815fca0 0x7f35780b3e50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35780b5c90 0x7f3578157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 --2- 172.21.15.27:0/145822268 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f35780b6110 0x7f35780b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 >> 172.21.15.27:0/145822268 conn(0x7f357806c150 msgr2=0x7f35780cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 shutdown_connections 2023-07-06T20:15:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.609+0000 7f357e7ff700 1 -- 172.21.15.27:0/145822268 wait complete. 2023-07-06T20:15:05.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T20:15:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.839+0000 7fb649bff700 1 Processor -- start 2023-07-06T20:15:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.839+0000 7fb649bff700 1 -- start start 2023-07-06T20:15:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb649bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb6440cfe40 0x7fb6440cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb649bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6440b2180 0x7fb6440d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb649bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb644160cc0 con 0x7fb6440b8440 2023-07-06T20:15:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb649bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb644160ae0 con 0x7fb6440b2180 2023-07-06T20:15:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb649bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb644160f80 con 0x7fb6440cfe40 2023-07-06T20:15:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6440b2180 0x7fb6440d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:05.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6440b2180 0x7fb6440d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33404/0 (socket says 172.21.15.27:33404) 2023-07-06T20:15:05.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 -- 172.21.15.27:0/3566184564 learned_addr learned my addr 172.21.15.27:0/3566184564 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:05.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb642ffd700 1 --2- 172.21.15.27:0/3566184564 >> v2:172.21.15.153:3300/0 conn(0x7fb6440cfe40 0x7fb6440cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:05.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 -- 172.21.15.27:0/3566184564 >> v2:172.21.15.153:3300/0 conn(0x7fb6440cfe40 msgr2=0x7fb6440cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:05.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 --2- 172.21.15.27:0/3566184564 >> v2:172.21.15.153:3300/0 conn(0x7fb6440cfe40 0x7fb6440cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 -- 172.21.15.27:0/3566184564 >> v1:172.21.15.153:6789/0 conn(0x7fb6440b8440 legacy=0x7fb6440cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:05.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.841+0000 7fb643fff700 1 -- 172.21.15.27:0/3566184564 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440cf0e0 con 0x7fb6440b2180 2023-07-06T20:15:05.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.842+0000 7fb643fff700 1 --2- 172.21.15.27:0/3566184564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440b2180 0x7fb6440d0960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6440b1bc0 tx=0x7fb63c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2003955b16796eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:05.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.843+0000 7fb6427fc700 1 -- 172.21.15.27:0/3566184564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00e070 con 0x7fb6440b2180 2023-07-06T20:15:05.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.843+0000 7fb6427fc700 1 -- 172.21.15.27:0/3566184564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c00b910 con 0x7fb6440b2180 2023-07-06T20:15:05.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.843+0000 7fb6427fc700 1 -- 172.21.15.27:0/3566184564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00bcc0 con 0x7fb6440b2180 2023-07-06T20:15:05.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 -- 172.21.15.27:0/3566184564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440b2180 msgr2=0x7fb6440d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:05.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 --2- 172.21.15.27:0/3566184564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440b2180 0x7fb6440d0960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6440b1bc0 tx=0x7fb63c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:05.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 -- 172.21.15.27:0/3566184564 shutdown_connections 2023-07-06T20:15:05.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 --2- 172.21.15.27:0/3566184564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440b2180 0x7fb6440d0960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 --2- 172.21.15.27:0/3566184564 >> v2:172.21.15.153:3300/0 conn(0x7fb6440cfe40 0x7fb6440cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 -- 172.21.15.27:0/3566184564 >> 172.21.15.27:0/3566184564 conn(0x7fb64406be10 msgr2=0x7fb644071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:05.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 -- 172.21.15.27:0/3566184564 shutdown_connections 2023-07-06T20:15:05.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.844+0000 7fb649bff700 1 -- 172.21.15.27:0/3566184564 wait complete. 2023-07-06T20:15:05.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.846+0000 7fb649bff700 1 Processor -- start 2023-07-06T20:15:05.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.846+0000 7fb649bff700 1 -- start start 2023-07-06T20:15:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb649bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6440b2180 0x7fb644149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb649bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6440b8440 0x7fb644157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb649bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 0x7fb644159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb649bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440c7450 con 0x7fb6440cfe40 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb649bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb644149760 con 0x7fb6440b8440 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb649bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440c7870 con 0x7fb6440b2180 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb643fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 0x7fb644159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb643fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 0x7fb644159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33412/0 (socket says 172.21.15.27:33412) 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.847+0000 7fb643fff700 1 -- 172.21.15.27:0/3051381489 learned_addr learned my addr 172.21.15.27:0/3051381489 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb642ffd700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6440b8440 0x7fb644157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:05.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb643fff700 1 -- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6440b2180 msgr2=0x7fb644149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:05.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb6437fe700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6440b2180 0x7fb644149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:05.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb643fff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6440b2180 0x7fb644149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb643fff700 1 -- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6440b8440 msgr2=0x7fb644157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:05.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb643fff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6440b8440 0x7fb644157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:05.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb643fff700 1 -- 172.21.15.27:0/3051381489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c009b30 con 0x7fb6440cfe40 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb642ffd700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6440b8440 0x7fb644157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb6437fe700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6440b2180 0x7fb644149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb643fff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 0x7fb644159120 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6440d0690 tx=0x7fb63c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0099b0 con 0x7fb6440cfe40 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c0092e0 con 0x7fb6440cfe40 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c01b630 con 0x7fb6440cfe40 2023-07-06T20:15:05.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.848+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6440c6860 con 0x7fb6440cfe40 2023-07-06T20:15:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.849+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6440c6d90 con 0x7fb6440cfe40 2023-07-06T20:15:05.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.850+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6440c7590 con 0x7fb6440cfe40 2023-07-06T20:15:05.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.851+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb63c01b790 con 0x7fb6440cfe40 2023-07-06T20:15:05.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.852+0000 7fb62bfff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63006ace0 0x7fb63006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:05.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.852+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fb63c08ae70 con 0x7fb6440cfe40 2023-07-06T20:15:05.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.852+0000 7fb6437fe700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63006ace0 0x7fb63006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:05.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.853+0000 7fb6437fe700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63006ace0 0x7fb63006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6441564b0 tx=0x7fb634009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:05.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:05.856+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb6440c7590 con 0x7fb6440cfe40 2023-07-06T20:15:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.067+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6440c7590 con 0x7fb6440cfe40 2023-07-06T20:15:06.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.846+0000 7fb62bfff700 1 -- 172.21.15.27:0/3051381489 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6440c7590 con 0x7fb6440cfe40 2023-07-06T20:15:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.849+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63006ace0 msgr2=0x7fb63006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.849+0000 7fb649bff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63006ace0 0x7fb63006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6441564b0 tx=0x7fb634009040 comp rx=0 tx=0).stop 2023-07-06T20:15:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.849+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 msgr2=0x7fb644159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.849+0000 7fb649bff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 0x7fb644159120 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6440d0690 tx=0x7fb63c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.849+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 shutdown_connections 2023-07-06T20:15:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63006ace0 0x7fb63006d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6440cfe40 0x7fb644159120 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6440b2180 0x7fb644149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 --2- 172.21.15.27:0/3051381489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6440b8440 0x7fb644157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 >> 172.21.15.27:0/3051381489 conn(0x7fb64406be10 msgr2=0x7fb6440b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 shutdown_connections 2023-07-06T20:15:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:06.850+0000 7fb649bff700 1 -- 172.21.15.27:0/3051381489 wait complete. 2023-07-06T20:15:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-07-06T20:15:06.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T20:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.083+0000 7f675fbff700 1 Processor -- start 2023-07-06T20:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.083+0000 7f675fbff700 1 -- start start 2023-07-06T20:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675fbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f67580cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675fbff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f67580b2180 0x7f67580d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675fbff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6758160cc0 con 0x7f67580b8440 2023-07-06T20:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675fbff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6758160ae0 con 0x7f67580b2180 2023-07-06T20:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675fbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6758160f80 con 0x7f67580cfe40 2023-07-06T20:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675d99b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f67580b8440 0x7f67580cf730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53058/0 (socket says 172.21.15.27:53058) 2023-07-06T20:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675d99b700 1 -- 172.21.15.27:0/650663218 learned_addr learned my addr 172.21.15.27:0/650663218 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675d19a700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f67580cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.084+0000 7f675e19c700 1 --2- 172.21.15.27:0/650663218 >> v2:172.21.15.153:3300/0 conn(0x7f67580b2180 0x7f67580d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123933687 0 0) 0x7f6758160cc0 con 0x7f67580b8440 2023-07-06T20:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67440035b0 con 0x7f67580b8440 2023-07-06T20:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086449247 0 0) 0x7f67440035b0 con 0x7f67580b8440 2023-07-06T20:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 msgr2=0x7f67580cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f67580cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 >> v2:172.21.15.153:3300/0 conn(0x7f67580b2180 msgr2=0x7f67580d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 --2- 172.21.15.27:0/650663218 >> v2:172.21.15.153:3300/0 conn(0x7f67580b2180 0x7f67580d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580cf0e0 con 0x7f67580b8440 2023-07-06T20:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f6754001670 con 0x7f67580b8440 2023-07-06T20:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675e19c700 1 --2- 172.21.15.27:0/650663218 >> v2:172.21.15.153:3300/0 conn(0x7f67580b2180 0x7f67580d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 >> v1:172.21.15.27:6789/0 conn(0x7f67580b8440 legacy=0x7f67580cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744003a00 0x7f6744003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6744004850 0x7f6744004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f6744005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758160ae0 con 0x7f6744004850 2023-07-06T20:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758160f80 con 0x7f6744003a00 2023-07-06T20:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674400ba10 con 0x7f67580cfe40 2023-07-06T20:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.085+0000 7f675d19a700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6744004850 0x7f6744004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d99b700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744003a00 0x7f6744003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675e19c700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f6744005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d19a700 1 -- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 msgr2=0x7f6744005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d19a700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f6744005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d19a700 1 -- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744003a00 msgr2=0x7f6744003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d19a700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744003a00 0x7f6744003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d19a700 1 -- 172.21.15.27:0/650663218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440035b0 con 0x7f6744004850 2023-07-06T20:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675e19c700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67580cfe40 0x7f6744005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675d19a700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6744004850 0x7f6744004cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6748009b60 tx=0x7f6744005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dcd0eddabff0f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800e6c0 con 0x7f6744004850 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674800e820 con 0x7f6744004850 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.086+0000 7f675c999700 1 -- 172.21.15.27:0/650663218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800eb90 con 0x7f6744004850 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675fbff700 1 -- 172.21.15.27:0/650663218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6744004850 msgr2=0x7f6744004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675fbff700 1 --2- 172.21.15.27:0/650663218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6744004850 0x7f6744004cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6748009b60 tx=0x7f6744005880 comp rx=0 tx=0).stop 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675e19c700 1 -- 172.21.15.27:0/650663218 reap_dead start 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675fbff700 1 -- 172.21.15.27:0/650663218 shutdown_connections 2023-07-06T20:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675fbff700 1 -- 172.21.15.27:0/650663218 >> 172.21.15.27:0/650663218 conn(0x7f675806be10 msgr2=0x7f6758071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675fbff700 1 -- 172.21.15.27:0/650663218 shutdown_connections 2023-07-06T20:15:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.087+0000 7f675fbff700 1 -- 172.21.15.27:0/650663218 wait complete. 2023-07-06T20:15:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.089+0000 7f675fbff700 1 Processor -- start 2023-07-06T20:15:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.089+0000 7f675fbff700 1 -- start start 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675fbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744004850 0x7f675814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675fbff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67581f2840 0x7f67581e8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675fbff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 0x7f67580cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675fbff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67581e6fc0 con 0x7f67581e85a0 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675fbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67581e72a0 con 0x7f6744004850 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675fbff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67581e7580 con 0x7f67581f2840 2023-07-06T20:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675e19c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 0x7f67580cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675e19c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 0x7f67580cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33420/0 (socket says 172.21.15.27:33420) 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.090+0000 7f675e19c700 1 -- 172.21.15.27:0/4077797930 learned_addr learned my addr 172.21.15.27:0/4077797930 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675d99b700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744004850 0x7f675814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675d19a700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67581f2840 0x7f67581e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675e19c700 1 -- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67581f2840 msgr2=0x7f67581e8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675e19c700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67581f2840 0x7f67581e8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675e19c700 1 -- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744004850 msgr2=0x7f675814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675e19c700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744004850 0x7f675814b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675e19c700 1 -- 172.21.15.27:0/4077797930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67581e7860 con 0x7f67581e85a0 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675d19a700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67581f2840 0x7f67581e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675e19c700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 0x7f67580cc2e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67580b1bc0 tx=0x7f67500116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000e070 con 0x7f67581e85a0 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6750011e50 con 0x7f67581e85a0 2023-07-06T20:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67581e7b40 con 0x7f67581e85a0 2023-07-06T20:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.091+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750003a20 con 0x7f67581e85a0 2023-07-06T20:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.092+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67581f2cb0 con 0x7f67581e85a0 2023-07-06T20:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.093+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67580cd8b0 con 0x7f67581e85a0 2023-07-06T20:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.093+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6750003b80 con 0x7f67581e85a0 2023-07-06T20:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.094+0000 7f674e7fc700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f673c06ace0 0x7f673c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.094+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f6750089b70 con 0x7f67581e85a0 2023-07-06T20:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.095+0000 7f675d99b700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f673c06ace0 0x7f673c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.096+0000 7f675d99b700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f673c06ace0 0x7f673c06d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f675815ec40 tx=0x7f6754001dd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.099+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6750054ca0 con 0x7f67581e85a0 2023-07-06T20:15:07.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:07.315+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67581e7d30 con 0x7f67581e85a0 2023-07-06T20:15:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.078+0000 7f674e7fc700 1 -- 172.21.15.27:0/4077797930 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f67500582c0 con 0x7f67581e85a0 2023-07-06T20:15:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f673c06ace0 msgr2=0x7f673c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f673c06ace0 0x7f673c06d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f675815ec40 tx=0x7f6754001dd0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 msgr2=0x7f67580cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 0x7f67580cc2e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67580b1bc0 tx=0x7f67500116e0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 shutdown_connections 2023-07-06T20:15:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f673c06ace0 0x7f673c06d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67581e85a0 0x7f67580cc2e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6744004850 0x7f675814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 --2- 172.21.15.27:0/4077797930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f67581f2840 0x7f67581e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 >> 172.21.15.27:0/4077797930 conn(0x7f675806be10 msgr2=0x7f67580b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.082+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 shutdown_connections 2023-07-06T20:15:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.083+0000 7f675fbff700 1 -- 172.21.15.27:0/4077797930 wait complete. 2023-07-06T20:15:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-07-06T20:15:08.108 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T20:15:08.108 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T20:15:08.109 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T20:15:08.109 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:15:08.109 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T20:15:08.109 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T20:15:08.109 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T20:15:08.110 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:15:08.110 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T20:15:08.110 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T20:15:08.110 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T20:15:08.111 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:15:08.111 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T20:15:08.111 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T20:15:08.111 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T20:15:08.111 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:15:08.124 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T20:15:08.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T20:15:08.125 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T20:15:08.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T20:15:08.128 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T20:15:08.128 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T20:15:08.129 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T20:15:08.129 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T20:15:08.170 INFO:tasks.ceph.mds.c:Started 2023-07-06T20:15:08.171 DEBUG:teuthology.parallel:result is None 2023-07-06T20:15:08.172 INFO:tasks.ceph.mds.b:Started 2023-07-06T20:15:08.173 DEBUG:teuthology.parallel:result is None 2023-07-06T20:15:08.174 INFO:tasks.ceph.mds.a:Started 2023-07-06T20:15:08.175 DEBUG:teuthology.parallel:result is None 2023-07-06T20:15:08.176 INFO:tasks.ceph.mds.d:Started 2023-07-06T20:15:08.176 DEBUG:teuthology.parallel:result is None 2023-07-06T20:15:08.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T20:15:08.323 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2023-07-06T20:15:08.324 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2023-07-06T20:15:08.333 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-07-06T20:15:08.333 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-07-06T20:15:08.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.393+0000 7f9e6d1ff700 1 Processor -- start 2023-07-06T20:15:08.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.394+0000 7f9e6d1ff700 1 -- start start 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6d1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e680b5c90 0x7f9e680d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6d1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9e680b60e0 0x7f9e680d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6d1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680cf780 con 0x7f9e6815fca0 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6d1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680cd490 con 0x7f9e680b60e0 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6d1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680cd770 con 0x7f9e680b5c90 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e66d9d700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9e6815fca0 0x7f9e680d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53064/0 (socket says 172.21.15.27:53064) 2023-07-06T20:15:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e66d9d700 1 -- 172.21.15.27:0/1190960709 learned_addr learned my addr 172.21.15.27:0/1190960709 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6759e700 1 --2- 172.21.15.27:0/1190960709 >> v2:172.21.15.153:3300/0 conn(0x7f9e680b60e0 0x7f9e680d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.395+0000 7f9e6659c700 1 --2- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e680b5c90 0x7f9e680d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e65d9b700 1 -- 172.21.15.27:0/1190960709 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337537486 0 0) 0x7f9e680cf780 con 0x7f9e6815fca0 2023-07-06T20:15:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e65d9b700 1 -- 172.21.15.27:0/1190960709 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e500035b0 con 0x7f9e6815fca0 2023-07-06T20:15:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e6759e700 1 -- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e680b5c90 msgr2=0x7f9e680d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e6759e700 1 --2- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e680b5c90 0x7f9e680d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e6759e700 1 -- 172.21.15.27:0/1190960709 >> v1:172.21.15.27:6789/0 conn(0x7f9e6815fca0 legacy=0x7f9e680d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e6759e700 1 -- 172.21.15.27:0/1190960709 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680cda50 con 0x7f9e680b60e0 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e6759e700 1 --2- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b60e0 0x7f9e680d3280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009d90 tx=0x7f9e6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db7e46f790d49bad server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e65d9b700 1 -- 172.21.15.27:0/1190960709 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000e070 con 0x7f9e680b60e0 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e65d9b700 1 -- 172.21.15.27:0/1190960709 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6000bb20 con 0x7f9e680b60e0 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.396+0000 7f9e65d9b700 1 -- 172.21.15.27:0/1190960709 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60012430 con 0x7f9e680b60e0 2023-07-06T20:15:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b60e0 msgr2=0x7f9e680d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b60e0 0x7f9e680d3280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009d90 tx=0x7f9e6000b330 comp rx=0 tx=0).stop 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/1190960709 shutdown_connections 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b60e0 0x7f9e680d3280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/1190960709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e680b5c90 0x7f9e680d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/1190960709 >> 172.21.15.27:0/1190960709 conn(0x7f9e6806c150 msgr2=0x7f9e6806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/1190960709 shutdown_connections 2023-07-06T20:15:08.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.397+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/1190960709 wait complete. 2023-07-06T20:15:08.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.399+0000 7f9e6d1ff700 1 Processor -- start 2023-07-06T20:15:08.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.399+0000 7f9e6d1ff700 1 -- start start 2023-07-06T20:15:08.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6d1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b5c90 0x7f9e68151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6d1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 0x7f9e680b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6d1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e6815fca0 0x7f9e680b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6d1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68156380 con 0x7f9e680b60e0 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6d1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68156660 con 0x7f9e680b5c90 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6d1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68156940 con 0x7f9e6815fca0 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 0x7f9e680b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 0x7f9e680b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33450/0 (socket says 172.21.15.27:33450) 2023-07-06T20:15:08.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e66d9d700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b5c90 0x7f9e68151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6759e700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e6815fca0 0x7f9e680b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.400+0000 7f9e6659c700 1 -- 172.21.15.27:0/3676359463 learned_addr learned my addr 172.21.15.27:0/3676359463 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6659c700 1 -- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e6815fca0 msgr2=0x7f9e680b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6659c700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e6815fca0 0x7f9e680b8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6659c700 1 -- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b5c90 msgr2=0x7f9e68151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6659c700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b5c90 0x7f9e68151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:08.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6659c700 1 -- 172.21.15.27:0/3676359463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60009a70 con 0x7f9e680b60e0 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6759e700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e6815fca0 0x7f9e680b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e66d9d700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b5c90 0x7f9e68151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6659c700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 0x7f9e680b8000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00e9b0 tx=0x7f9e5c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c009710 con 0x7f9e680b60e0 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c00ee50 con 0x7f9e680b60e0 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e681534c0 con 0x7f9e680b60e0 2023-07-06T20:15:08.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00ca50 con 0x7f9e680b60e0 2023-07-06T20:15:08.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.401+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68151ab0 con 0x7f9e680b60e0 2023-07-06T20:15:08.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.403+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e68153830 con 0x7f9e680b60e0 2023-07-06T20:15:08.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.404+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e5c00cbb0 con 0x7f9e680b60e0 2023-07-06T20:15:08.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.405+0000 7f9e4f7fe700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5406ace0 0x7f9e5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:08.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.405+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9e5c0891d0 con 0x7f9e680b60e0 2023-07-06T20:15:08.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.405+0000 7f9e66d9d700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5406ace0 0x7f9e5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:08.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.406+0000 7f9e66d9d700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5406ace0 0x7f9e5406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9e58007fd0 tx=0x7f9e58007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:08.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.409+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e5c057f90 con 0x7f9e680b60e0 2023-07-06T20:15:08.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:08.633+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9e680008d0 con 0x7f9e680b60e0 2023-07-06T20:15:09.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.087+0000 7f9e4f7fe700 1 -- 172.21.15.27:0/3676359463 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e5c00ce60 con 0x7f9e680b60e0 2023-07-06T20:15:09.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5406ace0 msgr2=0x7f9e5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5406ace0 0x7f9e5406d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9e58007fd0 tx=0x7f9e58007dc0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 msgr2=0x7f9e680b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 0x7f9e680b8000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00e9b0 tx=0x7f9e5c00ecc0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 shutdown_connections 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9e5406ace0 0x7f9e5406d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9e6815fca0 0x7f9e680b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e680b5c90 0x7f9e68151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 --2- 172.21.15.27:0/3676359463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e680b60e0 0x7f9e680b8000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.090+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 >> 172.21.15.27:0/3676359463 conn(0x7f9e6806c150 msgr2=0x7f9e680b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.091+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 shutdown_connections 2023-07-06T20:15:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.091+0000 7f9e6d1ff700 1 -- 172.21.15.27:0/3676359463 wait complete. 2023-07-06T20:15:09.095 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-07-06T20:15:09.113 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T20:15:09.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.315+0000 7fd050bff700 1 Processor -- start 2023-07-06T20:15:09.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.315+0000 7fd050bff700 1 -- start start 2023-07-06T20:15:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd050bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd04c06c330 0x7fd04c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd050bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd050bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c06cc40 con 0x7fd04c0c9010 2023-07-06T20:15:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd050bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c0c9a60 con 0x7fd04c06c330 2023-07-06T20:15:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd050bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c159b70 con 0x7fd04c0c9480 2023-07-06T20:15:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd04a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd04c06c330 0x7fd04c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd04a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd04c06c330 0x7fd04c06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33452/0 (socket says 172.21.15.27:33452) 2023-07-06T20:15:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.316+0000 7fd04a59c700 1 -- 172.21.15.27:0/2609970627 learned_addr learned my addr 172.21.15.27:0/2609970627 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:09.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04ad9d700 1 --2- 172.21.15.27:0/2609970627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04a59c700 1 -- 172.21.15.27:0/2609970627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 msgr2=0x7fd04c156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04a59c700 1 --2- 172.21.15.27:0/2609970627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04a59c700 1 -- 172.21.15.27:0/2609970627 >> v1:172.21.15.153:6789/0 conn(0x7fd04c0c9010 legacy=0x7fd04c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04a59c700 1 -- 172.21.15.27:0/2609970627 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c159e50 con 0x7fd04c06c330 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04a59c700 1 --2- 172.21.15.27:0/2609970627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c06c700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00bb90 tx=0x7fd03c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7107d71aaf587958 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.317+0000 7fd04959a700 1 -- 172.21.15.27:0/2609970627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00dd50 con 0x7fd04c06c330 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd04959a700 1 -- 172.21.15.27:0/2609970627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c010470 con 0x7fd04c06c330 2023-07-06T20:15:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd04959a700 1 -- 172.21.15.27:0/2609970627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c010800 con 0x7fd04c06c330 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 -- 172.21.15.27:0/2609970627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 msgr2=0x7fd04c06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 --2- 172.21.15.27:0/2609970627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c06c700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00bb90 tx=0x7fd03c00bea0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 -- 172.21.15.27:0/2609970627 shutdown_connections 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 --2- 172.21.15.27:0/2609970627 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 --2- 172.21.15.27:0/2609970627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c06c700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 -- 172.21.15.27:0/2609970627 >> 172.21.15.27:0/2609970627 conn(0x7fd04c0cb440 msgr2=0x7fd04c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 -- 172.21.15.27:0/2609970627 shutdown_connections 2023-07-06T20:15:09.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.318+0000 7fd050bff700 1 -- 172.21.15.27:0/2609970627 wait complete. 2023-07-06T20:15:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.320+0000 7fd050bff700 1 Processor -- start 2023-07-06T20:15:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.320+0000 7fd050bff700 1 -- start start 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd050bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd050bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd04c0c9010 0x7fd04c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd050bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd050bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0b4780 con 0x7fd04c06c330 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd050bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1552b0 con 0x7fd04c0c9010 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd050bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0b0690 con 0x7fd04c0c9480 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.321+0000 7fd04a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c0b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33454/0 (socket says 172.21.15.27:33454) 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 -- 172.21.15.27:0/2624760481 learned_addr learned my addr 172.21.15.27:0/2624760481 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 -- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 msgr2=0x7fd04c070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 -- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd04c0c9010 msgr2=0x7fd04c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd04c0c9010 0x7fd04c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 -- 172.21.15.27:0/2624760481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c00b7f0 con 0x7fd04c06c330 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd04a59c700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c0b4080 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00bee0 tx=0x7fd03c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c01f070 con 0x7fd04c06c330 2023-07-06T20:15:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c0038b0 con 0x7fd04c06c330 2023-07-06T20:15:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c0b0970 con 0x7fd04c06c330 2023-07-06T20:15:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.322+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c003c80 con 0x7fd04c06c330 2023-07-06T20:15:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.323+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c0b0f30 con 0x7fd04c06c330 2023-07-06T20:15:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.323+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c0008d0 con 0x7fd04c06c330 2023-07-06T20:15:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.324+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd03c019400 con 0x7fd04c06c330 2023-07-06T20:15:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.325+0000 7fd03affd700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd02c06ace0 0x7fd02c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.325+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd03c08cd60 con 0x7fd04c06c330 2023-07-06T20:15:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.326+0000 7fd049d9b700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd02c06ace0 0x7fd02c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.327+0000 7fd049d9b700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd02c06ace0 0x7fd02c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f60 tx=0x7fd040007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:09.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.330+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd03c058200 con 0x7fd04c06c330 2023-07-06T20:15:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.619+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd04c15c4c0 con 0x7fd04c06c330 2023-07-06T20:15:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.620+0000 7fd03affd700 1 -- 172.21.15.27:0/2624760481 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd03c05b820 con 0x7fd04c06c330 2023-07-06T20:15:09.623 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:09.623 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:09.623 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-07-06T20:15:09.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.623 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-07-06T20:15:09.623 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBFIKdkt1qyIhAAV2Zt+4H6JHNHYPqFB0XaMQ==", 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBGIKdkCnGUARAAvSNtHM2BJDXHrEB3lIgd2Q==", 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBFIKdkXnKqLhAAXTskx14TZ4a4QPMiZi40cw==", 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-07-06T20:15:09.627 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBGIKdkJQMdDRAA8qma6ry2Uk87MqanA8+wNA==", 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBKIKdkDOsHDxAAk6iJhzgWm0nC2jkg5xqYPg==", 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBNIKdk3p88FBAAxfAk8Bh/1X5hyaAsauDmDA==", 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBQIKdk7yRFGBAAwSBhNBHWe6LAim+0MEhLqg==", 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.631 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBTIKdkLkbzHBAAaehxTk2QxX+ezYStqt6jPQ==", 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.632 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBZIKdkEcr+NRAAkb0WVzKF+o/k3L3Hm8k/Tg==", 2023-07-06T20:15:09.633 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-07-06T20:15:09.634 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBcIKdkU0Z0ORAA6RcJRYFnzuyoySt1Jws6Eg==", 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.635 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBgIKdkI9M+ARAAiyzZQS7rS182eKUu9hNazA==", 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBjIKdkDPf6BBAAEBZ8o1wlrgXFk0zD5oy0aQ==", 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBGIKdkSs+yFxAABbJ2vpRTPGJa6RY1/Ha9/Q==", 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-07-06T20:15:09.638 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBGIKdkIZgrJxAALjLhrHI8h1rmxJdwrYjp0g==", 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.639 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-07-06T20:15:09.640 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBGIKdkeD+IHxAAwuy2BoFocQRSfJX1NcVCZw==", 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBGIKdkscKnLhAAdw7o+z02ZMNZ+MGucPmEsw==", 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-07-06T20:15:09.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBEIKdkIH3HJRAA756eIQKU/jwD5LhyGCdPRw==", 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-07-06T20:15:09.643 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mds", 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBrIKdkjuB9ARAAWMy8MKt0UhUf2uphPshx0g==", 2023-07-06T20:15:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mds" 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mgr", 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBrIKdkgOx9ARAAkfWZ38JJuU1wTLpGyewz8w==", 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-osd", 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBrIKdkEvZ9ARAAPGPhasx+YRSbFiRN/8MVlQ==", 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-osd" 2023-07-06T20:15:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd", 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBrIKdkfP99ARAA+zi1Sby1nY4fqu3OXTlESw==", 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBrIKdkNwl+ARAAkLFi3zULhl3jXEvwgYq4YA==", 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rgw", 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBrIKdk6hJ+ARAAX6Bj/kcMsrYLGNpf8WLJDQ==", 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.649 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBFIKdkigi+GhAAKT/PZgsjm5sUIw5t3MXX1w==", 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQBFIKdkfs6mEhAARTSorZ0o/g9w2V6Dr7ArBg==", 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-07-06T20:15:09.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:09.652 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-07-06T20:15:09.652 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:09.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd02c06ace0 msgr2=0x7fd02c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd02c06ace0 0x7fd02c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f60 tx=0x7fd040007a40 comp rx=0 tx=0).stop 2023-07-06T20:15:09.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 msgr2=0x7fd04c0b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c0b4080 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00bee0 tx=0x7fd03c004750 comp rx=0 tx=0).stop 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 shutdown_connections 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd02c06ace0 0x7fd02c06d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd04c0c9480 0x7fd04c070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd04c06c330 0x7fd04c0b4080 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 --2- 172.21.15.27:0/2624760481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd04c0c9010 0x7fd04c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 >> 172.21.15.27:0/2624760481 conn(0x7fd04c0cb440 msgr2=0x7fd04c15cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 shutdown_connections 2023-07-06T20:15:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.622+0000 7fd050bff700 1 -- 172.21.15.27:0/2624760481 wait complete. 2023-07-06T20:15:09.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-06T20:15:09.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.857+0000 7f480bdff700 1 Processor -- start 2023-07-06T20:15:09.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.857+0000 7f480bdff700 1 -- start start 2023-07-06T20:15:09.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.858+0000 7f480bdff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f480415fca0 0x7f48040d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.858+0000 7f480bdff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040cf780 con 0x7f48040b6110 2023-07-06T20:15:09.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.858+0000 7f480bdff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040cd490 con 0x7f48040b5c90 2023-07-06T20:15:09.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.858+0000 7f480bdff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040cd770 con 0x7f480415fca0 2023-07-06T20:15:09.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.859+0000 7f480a39c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f48040b6110 0x7f48040cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53072/0 (socket says 172.21.15.27:53072) 2023-07-06T20:15:09.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.859+0000 7f480a39c700 1 -- 172.21.15.27:0/2385267762 learned_addr learned my addr 172.21.15.27:0/2385267762 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:09.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.859+0000 7f4809b9b700 1 --2- 172.21.15.27:0/2385267762 >> v2:172.21.15.153:3300/0 conn(0x7f480415fca0 0x7f48040d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.859+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968259580 0 0) 0x7f48040cf780 con 0x7f48040b6110 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.859+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f00035b0 con 0x7f48040b6110 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.859+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917040465 0 0) 0x7f48040cd490 con 0x7f48040b5c90 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48040cf780 con 0x7f48040b5c90 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941954903 0 0) 0x7f47f00035b0 con 0x7f48040b6110 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 >> v2:172.21.15.153:3300/0 conn(0x7f480415fca0 msgr2=0x7f48040d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 --2- 172.21.15.27:0/2385267762 >> v2:172.21.15.153:3300/0 conn(0x7f480415fca0 0x7f48040d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 >> v1:172.21.15.153:6789/0 conn(0x7f48040b5c90 legacy=0x7f48040d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040cda50 con 0x7f48040b6110 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f47fc0019f0 con 0x7f48040b6110 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 >> v1:172.21.15.27:6789/0 conn(0x7f48040b6110 legacy=0x7f48040cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f0003be0 0x7f47f0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f0004850 0x7f47f0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0005200 0x7f47f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040cd770 con 0x7f47f0004850 2023-07-06T20:15:09.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f000c0e0 con 0x7f47f0003be0 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f000c270 con 0x7f47f0005200 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480939a700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f0004850 0x7f47f0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f4809b9b700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f0003be0 0x7f47f0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480a39c700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0005200 0x7f47f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480939a700 1 -- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0005200 msgr2=0x7f47f000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480939a700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0005200 0x7f47f000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480939a700 1 -- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f0003be0 msgr2=0x7f47f0003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:09.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480939a700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47f0003be0 0x7f47f0003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480939a700 1 -- 172.21.15.27:0/2385267762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00035b0 con 0x7f47f0004850 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.860+0000 7f480a39c700 1 -- 172.21.15.27:0/2385267762 reap_dead start 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.861+0000 7f480939a700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f0004850 0x7f47f0004cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f47f4007fd0 tx=0x7f47f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b177ff8ab47680c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.861+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4001430 con 0x7f47f0004850 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.861+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f40052f0 con 0x7f47f0004850 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.861+0000 7f4808b99700 1 -- 172.21.15.27:0/2385267762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4001d40 con 0x7f47f0004850 2023-07-06T20:15:09.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.861+0000 7f480bdff700 1 -- 172.21.15.27:0/2385267762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f0004850 msgr2=0x7f47f0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.861+0000 7f480bdff700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f0004850 0x7f47f0004cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f47f4007fd0 tx=0x7f47f0005880 comp rx=0 tx=0).stop 2023-07-06T20:15:09.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.862+0000 7f480bdff700 1 -- 172.21.15.27:0/2385267762 shutdown_connections 2023-07-06T20:15:09.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.862+0000 7f480bdff700 1 --2- 172.21.15.27:0/2385267762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f0004850 0x7f47f0004cc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.862+0000 7f480bdff700 1 -- 172.21.15.27:0/2385267762 >> 172.21.15.27:0/2385267762 conn(0x7f480406c150 msgr2=0x7f480406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:09.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.862+0000 7f480bdff700 1 -- 172.21.15.27:0/2385267762 shutdown_connections 2023-07-06T20:15:09.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.862+0000 7f480bdff700 1 -- 172.21.15.27:0/2385267762 wait complete. 2023-07-06T20:15:09.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.864+0000 7f480bdff700 1 Processor -- start 2023-07-06T20:15:09.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.864+0000 7f480bdff700 1 -- start start 2023-07-06T20:15:09.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480bdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0004850 0x7f48040c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480bdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48040c9f10 0x7f4804157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480bdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 0x7f48040cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480bdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040ca440 con 0x7f48040cb4e0 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480bdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040ca720 con 0x7f48040c9f10 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480bdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040caa00 con 0x7f47f0004850 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 0x7f48040cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 0x7f48040cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33480/0 (socket says 172.21.15.27:33480) 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 -- 172.21.15.27:0/3255106951 learned_addr learned my addr 172.21.15.27:0/3255106951 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:09.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480939a700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48040c9f10 0x7f4804157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f4809b9b700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0004850 0x7f48040c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 -- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0004850 msgr2=0x7f48040c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0004850 0x7f48040c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 -- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48040c9f10 msgr2=0x7f4804157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48040c9f10 0x7f4804157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.866+0000 7f480a39c700 1 -- 172.21.15.27:0/3255106951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040cace0 con 0x7f48040cb4e0 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.867+0000 7f480a39c700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 0x7f48040cbb70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f48040cf6f0 tx=0x7f47fc007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:09.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.867+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00eb40 con 0x7f48040cb4e0 2023-07-06T20:15:09.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.867+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00eca0 con 0x7f48040cb4e0 2023-07-06T20:15:09.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.867+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48040cafc0 con 0x7f48040cb4e0 2023-07-06T20:15:09.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.867+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc013420 con 0x7f48040cb4e0 2023-07-06T20:15:09.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.867+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48041577d0 con 0x7f48040cb4e0 2023-07-06T20:15:09.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.868+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48040cb1b0 con 0x7f48040cb4e0 2023-07-06T20:15:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.869+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47fc0135a0 con 0x7f48040cb4e0 2023-07-06T20:15:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.871+0000 7f47fa7fc700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f47e806ace0 0x7f47e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.871+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f47fc089610 con 0x7f48040cb4e0 2023-07-06T20:15:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.872+0000 7f4809b9b700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f47e806ace0 0x7f47e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:09.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.872+0000 7f4809b9b700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f47e806ace0 0x7f47e806d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c00 tx=0x7f4800009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:09.875+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f47fc054970 con 0x7f48040cb4e0 2023-07-06T20:15:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.172+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f48040008d0 con 0x7f48040cb4e0 2023-07-06T20:15:10.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.175+0000 7f47fa7fc700 1 -- 172.21.15.27:0/3255106951 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f47fc057f90 con 0x7f48040cb4e0 2023-07-06T20:15:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f47e806ace0 msgr2=0x7f47e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f47e806ace0 0x7f47e806d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c00 tx=0x7f4800009210 comp rx=0 tx=0).stop 2023-07-06T20:15:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 msgr2=0x7f48040cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 0x7f48040cbb70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f48040cf6f0 tx=0x7f47fc007c90 comp rx=0 tx=0).stop 2023-07-06T20:15:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 shutdown_connections 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f47e806ace0 0x7f47e806d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48040cb4e0 0x7f48040cbb70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47f0004850 0x7f48040c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 --2- 172.21.15.27:0/3255106951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48040c9f10 0x7f4804157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 >> 172.21.15.27:0/3255106951 conn(0x7f480406c150 msgr2=0x7f48040b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 shutdown_connections 2023-07-06T20:15:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.178+0000 7f480bdff700 1 -- 172.21.15.27:0/3255106951 wait complete. 2023-07-06T20:15:10.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-06T20:15:10.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.401+0000 7fd6f13ff700 1 Processor -- start 2023-07-06T20:15:10.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.401+0000 7fd6f13ff700 1 -- start start 2023-07-06T20:15:10.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.402+0000 7fd6f13ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd6ec15fca0 0x7fd6ec0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.402+0000 7fd6f13ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6ec0b6110 0x7fd6ec0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.402+0000 7fd6f13ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0cf780 con 0x7fd6ec0b5c90 2023-07-06T20:15:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.402+0000 7fd6f13ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0cd490 con 0x7fd6ec0b6110 2023-07-06T20:15:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.402+0000 7fd6f13ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0cd770 con 0x7fd6ec15fca0 2023-07-06T20:15:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6ec0b6110 0x7fd6ec0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6ec0b6110 0x7fd6ec0cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33488/0 (socket says 172.21.15.27:33488) 2023-07-06T20:15:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/262999827 learned_addr learned my addr 172.21.15.27:0/262999827 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eaffd700 1 --2- 172.21.15.27:0/262999827 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec15fca0 0x7fd6ec0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/262999827 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec15fca0 msgr2=0x7fd6ec0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 --2- 172.21.15.27:0/262999827 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec15fca0 0x7fd6ec0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/262999827 >> v1:172.21.15.153:6789/0 conn(0x7fd6ec0b5c90 legacy=0x7fd6ec0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/262999827 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0cda50 con 0x7fd6ec0b6110 2023-07-06T20:15:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.403+0000 7fd6eb7fe700 1 --2- 172.21.15.27:0/262999827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec0b6110 0x7fd6ec0cfc10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0d0930 tx=0x7fd6e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aecf65e4a92528f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6e9ffb700 1 -- 172.21.15.27:0/262999827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400e070 con 0x7fd6ec0b6110 2023-07-06T20:15:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6e9ffb700 1 -- 172.21.15.27:0/262999827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e400b9c0 con 0x7fd6ec0b6110 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6e9ffb700 1 -- 172.21.15.27:0/262999827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400bd70 con 0x7fd6ec0b6110 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 -- 172.21.15.27:0/262999827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec0b6110 msgr2=0x7fd6ec0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/262999827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec0b6110 0x7fd6ec0cfc10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0d0930 tx=0x7fd6e400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 -- 172.21.15.27:0/262999827 shutdown_connections 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/262999827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec0b6110 0x7fd6ec0cfc10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/262999827 >> v2:172.21.15.153:3300/0 conn(0x7fd6ec15fca0 0x7fd6ec0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 -- 172.21.15.27:0/262999827 >> 172.21.15.27:0/262999827 conn(0x7fd6ec06c150 msgr2=0x7fd6ec06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:10.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 -- 172.21.15.27:0/262999827 shutdown_connections 2023-07-06T20:15:10.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.404+0000 7fd6f13ff700 1 -- 172.21.15.27:0/262999827 wait complete. 2023-07-06T20:15:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.406+0000 7fd6f13ff700 1 Processor -- start 2023-07-06T20:15:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.406+0000 7fd6f13ff700 1 -- start start 2023-07-06T20:15:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6f13ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b5c90 0x7fd6ec1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6f13ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec0b6110 0x7fd6ec158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6f13ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 0x7fd6ec159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6f13ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec157c10 con 0x7fd6ec15fca0 2023-07-06T20:15:10.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6f13ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0c81c0 con 0x7fd6ec0b5c90 2023-07-06T20:15:10.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6f13ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0c84a0 con 0x7fd6ec0b6110 2023-07-06T20:15:10.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6eb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 0x7fd6ec159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6eb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 0x7fd6ec159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33498/0 (socket says 172.21.15.27:33498) 2023-07-06T20:15:10.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.407+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/740225486 learned_addr learned my addr 172.21.15.27:0/740225486 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:10.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eaffd700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b5c90 0x7fd6ec1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6ea7fc700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec0b6110 0x7fd6ec158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec0b6110 msgr2=0x7fd6ec158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eb7fe700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec0b6110 0x7fd6ec158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b5c90 msgr2=0x7fd6ec1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eb7fe700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b5c90 0x7fd6ec1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eb7fe700 1 -- 172.21.15.27:0/740225486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4009b30 con 0x7fd6ec15fca0 2023-07-06T20:15:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6ea7fc700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec0b6110 0x7fd6ec158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6eb7fe700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 0x7fd6ec159530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06d0f0 tx=0x7fd6e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40099b0 con 0x7fd6ec15fca0 2023-07-06T20:15:10.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e40092e0 con 0x7fd6ec15fca0 2023-07-06T20:15:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e401c640 con 0x7fd6ec15fca0 2023-07-06T20:15:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.408+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec1f5620 con 0x7fd6ec15fca0 2023-07-06T20:15:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.409+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec1f5be0 con 0x7fd6ec15fca0 2023-07-06T20:15:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.410+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec1f5480 con 0x7fd6ec15fca0 2023-07-06T20:15:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.411+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6e401c7a0 con 0x7fd6ec15fca0 2023-07-06T20:15:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.412+0000 7fd6d37fe700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd6d806ace0 0x7fd6d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.412+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd6e408ab70 con 0x7fd6ec15fca0 2023-07-06T20:15:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.412+0000 7fd6eaffd700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd6d806ace0 0x7fd6d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.413+0000 7fd6eaffd700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd6d806ace0 0x7fd6d806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00b9b0 tx=0x7fd6dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:10.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.416+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd6ec1f5480 con 0x7fd6ec15fca0 2023-07-06T20:15:10.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.707+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd6ec1f5480 con 0x7fd6ec15fca0 2023-07-06T20:15:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.712+0000 7fd6d37fe700 1 -- 172.21.15.27:0/740225486 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd6ec1f5480 con 0x7fd6ec15fca0 2023-07-06T20:15:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd6d806ace0 msgr2=0x7fd6d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd6d806ace0 0x7fd6d806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00b9b0 tx=0x7fd6dc008040 comp rx=0 tx=0).stop 2023-07-06T20:15:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 msgr2=0x7fd6ec159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 0x7fd6ec159530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06d0f0 tx=0x7fd6e4004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 shutdown_connections 2023-07-06T20:15:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd6d806ace0 0x7fd6d806d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6ec15fca0 0x7fd6ec159530 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6ec0b5c90 0x7fd6ec1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 --2- 172.21.15.27:0/740225486 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd6ec0b6110 0x7fd6ec158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 >> 172.21.15.27:0/740225486 conn(0x7fd6ec06c150 msgr2=0x7fd6ec0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:10.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 shutdown_connections 2023-07-06T20:15:10.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.714+0000 7fd6f13ff700 1 -- 172.21.15.27:0/740225486 wait complete. 2023-07-06T20:15:10.734 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-06T20:15:10.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 Processor -- start 2023-07-06T20:15:10.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 -- start start 2023-07-06T20:15:10.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b0415fca0 0x7f1b040d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b6110 0x7f1b040cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b040cf780 con 0x7f1b040b5c90 2023-07-06T20:15:10.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b040cd490 con 0x7f1b0415fca0 2023-07-06T20:15:10.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040cd770 con 0x7f1b040b6110 2023-07-06T20:15:10.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.937+0000 7f1b0979b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b0415fca0 0x7f1b040d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b0415fca0 0x7f1b040d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33514/0 (socket says 172.21.15.27:33514) 2023-07-06T20:15:10.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 -- 172.21.15.27:0/1672957281 learned_addr learned my addr 172.21.15.27:0/1672957281 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:10.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b09f9c700 1 --2- 172.21.15.27:0/1672957281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b6110 0x7f1b040cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 -- 172.21.15.27:0/1672957281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b6110 msgr2=0x7f1b040cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:10.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 --2- 172.21.15.27:0/1672957281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b6110 0x7f1b040cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 -- 172.21.15.27:0/1672957281 >> v1:172.21.15.153:6789/0 conn(0x7f1b040b5c90 legacy=0x7f1b040d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 -- 172.21.15.27:0/1672957281 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040cda50 con 0x7f1b0415fca0 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.938+0000 7f1b0979b700 1 --2- 172.21.15.27:0/1672957281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b040d1420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b000098f0 tx=0x7f1b00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8288a358a717d285 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1afbfff700 1 -- 172.21.15.27:0/1672957281 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000b8a0 con 0x7f1b0415fca0 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1afbfff700 1 -- 172.21.15.27:0/1672957281 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000ba00 con 0x7f1b0415fca0 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1afbfff700 1 -- 172.21.15.27:0/1672957281 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000bd90 con 0x7f1b0415fca0 2023-07-06T20:15:10.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/1672957281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 msgr2=0x7f1b040d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/1672957281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b040d1420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b000098f0 tx=0x7f1b00009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/1672957281 shutdown_connections 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/1672957281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b6110 0x7f1b040cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/1672957281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b040d1420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/1672957281 >> 172.21.15.27:0/1672957281 conn(0x7f1b0406c150 msgr2=0x7f1b0406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/1672957281 shutdown_connections 2023-07-06T20:15:10.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.939+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/1672957281 wait complete. 2023-07-06T20:15:10.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.942+0000 7f1b0b9ff700 1 Processor -- start 2023-07-06T20:15:10.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.942+0000 7f1b0b9ff700 1 -- start start 2023-07-06T20:15:10.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.943+0000 7f1b0b9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b5c90 0x7f1b040cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.943+0000 7f1b0b9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b040b6110 0x7f1b0415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.943+0000 7f1b0b9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b041f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.943+0000 7f1b0b9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c92b0 con 0x7f1b0415fca0 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.943+0000 7f1b0b9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c8390 con 0x7f1b040b6110 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.943+0000 7f1b0b9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c8630 con 0x7f1b040b5c90 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b041f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b041f11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33530/0 (socket says 172.21.15.27:33530) 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 -- 172.21.15.27:0/4113366490 learned_addr learned my addr 172.21.15.27:0/4113366490 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:10.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b0979b700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b5c90 0x7f1b040cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b08f9a700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b040b6110 0x7f1b0415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 -- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b5c90 msgr2=0x7f1b040cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b5c90 0x7f1b040cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 -- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b040b6110 msgr2=0x7f1b0415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:10.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b040b6110 0x7f1b0415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:10.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 -- 172.21.15.27:0/4113366490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00009550 con 0x7f1b0415fca0 2023-07-06T20:15:10.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.944+0000 7f1b09f9c700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b041f11a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1afc007e40 tx=0x7f1afc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:10.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.945+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00ed30 con 0x7f1b0415fca0 2023-07-06T20:15:10.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.945+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc014420 con 0x7f1b0415fca0 2023-07-06T20:15:10.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.945+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0147d0 con 0x7f1b0415fca0 2023-07-06T20:15:10.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.945+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b040c8910 con 0x7f1b0415fca0 2023-07-06T20:15:10.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.945+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b040c8ed0 con 0x7f1b0415fca0 2023-07-06T20:15:10.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.946+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b040008d0 con 0x7f1b0415fca0 2023-07-06T20:15:10.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.947+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1afc014930 con 0x7f1b0415fca0 2023-07-06T20:15:10.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.948+0000 7f1af9ffb700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ae806ace0 0x7f1ae806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:10.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.948+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1afc089790 con 0x7f1b0415fca0 2023-07-06T20:15:10.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.948+0000 7f1b0979b700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ae806ace0 0x7f1ae806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:10.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.949+0000 7f1b0979b700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ae806ace0 0x7f1ae806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b00014040 tx=0x7f1b00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:10.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:10.953+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1afc054cd0 con 0x7f1b0415fca0 2023-07-06T20:15:11.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.241+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f1b0415b8d0 con 0x7f1b0415fca0 2023-07-06T20:15:11.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.246+0000 7f1af9ffb700 1 -- 172.21.15.27:0/4113366490 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1afc0582f0 con 0x7f1b0415fca0 2023-07-06T20:15:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.248+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ae806ace0 msgr2=0x7f1ae806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.248+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ae806ace0 0x7f1ae806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b00014040 tx=0x7f1b00008030 comp rx=0 tx=0).stop 2023-07-06T20:15:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.248+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 msgr2=0x7f1b041f11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b041f11a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1afc007e40 tx=0x7f1afc00e3c0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 shutdown_connections 2023-07-06T20:15:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ae806ace0 0x7f1ae806d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b0415fca0 0x7f1b041f11a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b040b5c90 0x7f1b040cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 --2- 172.21.15.27:0/4113366490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b040b6110 0x7f1b0415dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 >> 172.21.15.27:0/4113366490 conn(0x7f1b0406c150 msgr2=0x7f1b040cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 shutdown_connections 2023-07-06T20:15:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.249+0000 7f1b0b9ff700 1 -- 172.21.15.27:0/4113366490 wait complete. 2023-07-06T20:15:11.270 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-06T20:15:11.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.472+0000 7f134a1ff700 1 Processor -- start 2023-07-06T20:15:11.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.472+0000 7f134a1ff700 1 -- start start 2023-07-06T20:15:11.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.473+0000 7f134a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f134415fca0 0x7f13440d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.473+0000 7f134a1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f13440b5c90 0x7f13440d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.473+0000 7f134a1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13440cf780 con 0x7f13440b6100 2023-07-06T20:15:11.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.473+0000 7f134a1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13440cd490 con 0x7f13440b5c90 2023-07-06T20:15:11.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.473+0000 7f134a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cd770 con 0x7f134415fca0 2023-07-06T20:15:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f1343fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f13440b6100 0x7f13440cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53086/0 (socket says 172.21.15.27:53086) 2023-07-06T20:15:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f1343fff700 1 -- 172.21.15.27:0/1326902645 learned_addr learned my addr 172.21.15.27:0/1326902645 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13437fe700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f134415fca0 0x7f13440d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f1342ffd700 1 --2- 172.21.15.27:0/1326902645 >> v2:172.21.15.153:3300/0 conn(0x7f13440b5c90 0x7f13440d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246203655 0 0) 0x7f13440cf780 con 0x7f13440b6100 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f132c0035b0 con 0x7f13440b6100 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378755718 0 0) 0x7f132c0035b0 con 0x7f13440b6100 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f134415fca0 msgr2=0x7f13440d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f134415fca0 0x7f13440d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 >> v2:172.21.15.153:3300/0 conn(0x7f13440b5c90 msgr2=0x7f13440d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 --2- 172.21.15.27:0/1326902645 >> v2:172.21.15.153:3300/0 conn(0x7f13440b5c90 0x7f13440d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440cda50 con 0x7f13440b6100 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.474+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f133c0019f0 con 0x7f13440b6100 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 >> v1:172.21.15.27:6789/0 conn(0x7f13440b6100 legacy=0x7f13440cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c003a00 0x7f132c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f132c004430 0x7f132c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f132c004f10 0x7f132c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cd490 con 0x7f132c004f10 2023-07-06T20:15:11.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cd770 con 0x7f132c003a00 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c00bf50 con 0x7f132c004430 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1342ffd700 1 --2- 172.21.15.27:0/1326902645 >> v2:172.21.15.153:3300/0 conn(0x7f13440b5c90 0x7f13440d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f13437fe700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f134415fca0 0x7f13440d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1343fff700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f132c004f10 0x7f132c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1343fff700 1 -- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f132c004430 msgr2=0x7f132c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1343fff700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f132c004430 0x7f132c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1343fff700 1 -- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c003a00 msgr2=0x7f132c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:11.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1343fff700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c003a00 0x7f132c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.475+0000 7f1343fff700 1 -- 172.21.15.27:0/1326902645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c0035b0 con 0x7f132c004f10 2023-07-06T20:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f1343fff700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f132c004f10 0x7f132c00ba10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f133c00a830 tx=0x7f132c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e6c278d4399220 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c012600 con 0x7f132c004f10 2023-07-06T20:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c012780 con 0x7f132c004f10 2023-07-06T20:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f13427fc700 1 -- 172.21.15.27:0/1326902645 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c012b30 con 0x7f132c004f10 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f134a1ff700 1 -- 172.21.15.27:0/1326902645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f132c004f10 msgr2=0x7f132c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f134a1ff700 1 --2- 172.21.15.27:0/1326902645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f132c004f10 0x7f132c00ba10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f133c00a830 tx=0x7f132c005880 comp rx=0 tx=0).stop 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f1343fff700 1 -- 172.21.15.27:0/1326902645 reap_dead start 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f134a1ff700 1 -- 172.21.15.27:0/1326902645 shutdown_connections 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f134a1ff700 1 -- 172.21.15.27:0/1326902645 >> 172.21.15.27:0/1326902645 conn(0x7f134406c150 msgr2=0x7f134406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.476+0000 7f134a1ff700 1 -- 172.21.15.27:0/1326902645 shutdown_connections 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.477+0000 7f134a1ff700 1 -- 172.21.15.27:0/1326902645 wait complete. 2023-07-06T20:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.478+0000 7f134a1ff700 1 Processor -- start 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.478+0000 7f134a1ff700 1 -- start start 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.479+0000 7f134a1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c004f10 0x7f13440aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.479+0000 7f134a1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 0x7f134415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.479+0000 7f134a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13440cc0a0 0x7f13440cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.479+0000 7f134a1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cca50 con 0x7f1344151100 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.479+0000 7f134a1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134415a6d0 con 0x7f132c004f10 2023-07-06T20:15:11.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.479+0000 7f134a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13440aa6d0 con 0x7f13440cc0a0 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 0x7f134415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 0x7f134415e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33542/0 (socket says 172.21.15.27:33542) 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 -- 172.21.15.27:0/2807683568 learned_addr learned my addr 172.21.15.27:0/2807683568 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f13437fe700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c004f10 0x7f13440aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1343fff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13440cc0a0 0x7f13440cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 -- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13440cc0a0 msgr2=0x7f13440cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13440cc0a0 0x7f13440cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 -- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c004f10 msgr2=0x7f13440aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c004f10 0x7f13440aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 -- 172.21.15.27:0/2807683568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440a9ac0 con 0x7f1344151100 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.480+0000 7f1342ffd700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 0x7f134415e2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f133800eb90 tx=0x7f133800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.481+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380097f0 con 0x7f1344151100 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.481+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338004030 con 0x7f1344151100 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.481+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338004420 con 0x7f1344151100 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.481+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13440a9da0 con 0x7f1344151100 2023-07-06T20:15:11.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.481+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13440aa360 con 0x7f1344151100 2023-07-06T20:15:11.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.482+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13440d27c0 con 0x7f1344151100 2023-07-06T20:15:11.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.483+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f133800c850 con 0x7f1344151100 2023-07-06T20:15:11.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.484+0000 7f132bfff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f133006ace0 0x7f133006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.484+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f13380890c0 con 0x7f1344151100 2023-07-06T20:15:11.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.484+0000 7f13437fe700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f133006ace0 0x7f133006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.485+0000 7f13437fe700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f133006ace0 0x7f133006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1334009470 tx=0x7f13340096d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:11.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.488+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13380544c0 con 0x7f1344151100 2023-07-06T20:15:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.774+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1344067000 con 0x7f1344151100 2023-07-06T20:15:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.778+0000 7f132bfff700 1 -- 172.21.15.27:0/2807683568 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1338057ae0 con 0x7f1344151100 2023-07-06T20:15:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f133006ace0 msgr2=0x7f133006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f133006ace0 0x7f133006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1334009470 tx=0x7f13340096d0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 msgr2=0x7f134415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 0x7f134415e2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f133800eb90 tx=0x7f133800eea0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 shutdown_connections 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f133006ace0 0x7f133006d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13440cc0a0 0x7f13440cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f132c004f10 0x7f13440aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 --2- 172.21.15.27:0/2807683568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344151100 0x7f134415e2e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 >> 172.21.15.27:0/2807683568 conn(0x7f134406c150 msgr2=0x7f13440b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 shutdown_connections 2023-07-06T20:15:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:11.780+0000 7f134a1ff700 1 -- 172.21.15.27:0/2807683568 wait complete. 2023-07-06T20:15:11.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-06T20:15:12.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.003+0000 7ff0963ff700 1 Processor -- start 2023-07-06T20:15:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.003+0000 7ff0963ff700 1 -- start start 2023-07-06T20:15:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff0963ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0900b2180 0x7ff0900ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff0963ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900d1370 con 0x7ff0900b8610 2023-07-06T20:15:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff0963ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900d1650 con 0x7ff0900cfe80 2023-07-06T20:15:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff0963ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0900cf410 con 0x7ff0900b2180 2023-07-06T20:15:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff08ffff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff0900b8610 0x7ff0900cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40614/0 (socket says 172.21.15.27:40614) 2023-07-06T20:15:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff08ffff700 1 -- 172.21.15.27:0/2010970020 learned_addr learned my addr 172.21.15.27:0/2010970020 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.004+0000 7ff09499c700 1 --2- 172.21.15.27:0/2010970020 >> v2:172.21.15.153:3300/0 conn(0x7ff0900b2180 0x7ff0900ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317628553 0 0) 0x7ff0900d1370 con 0x7ff0900b8610 2023-07-06T20:15:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0780035b0 con 0x7ff0900b8610 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156574943 0 0) 0x7ff0900d1650 con 0x7ff0900cfe80 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0900d1370 con 0x7ff0900cfe80 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3269885549 0 0) 0x7ff0780035b0 con 0x7ff0900b8610 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff09499c700 1 -- 172.21.15.27:0/2010970020 >> v1:172.21.15.153:6789/0 conn(0x7ff0900cfe80 legacy=0x7ff0900d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff09499c700 1 -- 172.21.15.27:0/2010970020 >> v1:172.21.15.27:6789/0 conn(0x7ff0900b8610 legacy=0x7ff0900cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff09499c700 1 -- 172.21.15.27:0/2010970020 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900b8cd0 con 0x7ff0900b2180 2023-07-06T20:15:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.005+0000 7ff09499c700 1 --2- 172.21.15.27:0/2010970020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff0900ceed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff088009d90 tx=0x7ff08800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=729ade076063c278 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800e070 con 0x7ff0900b2180 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088003a20 con 0x7ff0900b2180 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff08effd700 1 -- 172.21.15.27:0/2010970020 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0800015f0 con 0x7ff0900b2180 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 -- 172.21.15.27:0/2010970020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 msgr2=0x7ff0900ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 --2- 172.21.15.27:0/2010970020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff0900ceed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff088009d90 tx=0x7ff08800b330 comp rx=0 tx=0).stop 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 -- 172.21.15.27:0/2010970020 shutdown_connections 2023-07-06T20:15:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 --2- 172.21.15.27:0/2010970020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff0900ceed0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 -- 172.21.15.27:0/2010970020 >> 172.21.15.27:0/2010970020 conn(0x7ff090071690 msgr2=0x7ff090071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:12.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 -- 172.21.15.27:0/2010970020 shutdown_connections 2023-07-06T20:15:12.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.006+0000 7ff0963ff700 1 -- 172.21.15.27:0/2010970020 wait complete. 2023-07-06T20:15:12.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.008+0000 7ff0963ff700 1 Processor -- start 2023-07-06T20:15:12.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.008+0000 7ff0963ff700 1 -- start start 2023-07-06T20:15:12.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff0963ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff090156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff0963ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 0x7ff0900ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff0963ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0900cfe80 0x7ff09014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff0963ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901566a0 con 0x7ff0900b8610 2023-07-06T20:15:12.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff0963ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901571c0 con 0x7ff0900b2180 2023-07-06T20:15:12.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff0963ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901574a0 con 0x7ff0900cfe80 2023-07-06T20:15:12.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff08f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 0x7ff0900ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff08f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 0x7ff0900ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45436/0 (socket says 172.21.15.27:45436) 2023-07-06T20:15:12.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.009+0000 7ff08f7fe700 1 -- 172.21.15.27:0/3382273589 learned_addr learned my addr 172.21.15.27:0/3382273589 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:12.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff09499c700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0900cfe80 0x7ff09014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08ffff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff090156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08f7fe700 1 -- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0900cfe80 msgr2=0x7ff09014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08f7fe700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0900cfe80 0x7ff09014c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08f7fe700 1 -- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 msgr2=0x7ff090156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08f7fe700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff090156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08f7fe700 1 -- 172.21.15.27:0/3382273589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088009a70 con 0x7ff0900b8610 2023-07-06T20:15:12.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08ffff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff090156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:12.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08f7fe700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 0x7ff0900ca870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff084001eb0 tx=0x7ff08400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:12.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840052d0 con 0x7ff0900b8610 2023-07-06T20:15:12.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff084005450 con 0x7ff0900b8610 2023-07-06T20:15:12.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840038c0 con 0x7ff0900b8610 2023-07-06T20:15:12.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.010+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff090157780 con 0x7ff0900b8610 2023-07-06T20:15:12.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.011+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff090157d40 con 0x7ff0900b8610 2023-07-06T20:15:12.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.012+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0900d1ba0 con 0x7ff0900b8610 2023-07-06T20:15:12.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.013+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff084003a20 con 0x7ff0900b8610 2023-07-06T20:15:12.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.013+0000 7ff08cff9700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff07c06ad10 0x7ff07c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.014+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff084088600 con 0x7ff0900b8610 2023-07-06T20:15:12.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.014+0000 7ff08ffff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff07c06ad10 0x7ff07c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.014+0000 7ff08ffff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff07c06ad10 0x7ff07c06d1c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff0800078a0 tx=0x7ff080009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:12.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.018+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff0840539f0 con 0x7ff0900b8610 2023-07-06T20:15:12.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.316+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ff0900008d0 con 0x7ff0900b8610 2023-07-06T20:15:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.321+0000 7ff08cff9700 1 -- 172.21.15.27:0/3382273589 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff084057010 con 0x7ff0900b8610 2023-07-06T20:15:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff07c06ad10 msgr2=0x7ff07c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff07c06ad10 0x7ff07c06d1c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff0800078a0 tx=0x7ff080009040 comp rx=0 tx=0).stop 2023-07-06T20:15:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 msgr2=0x7ff0900ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 0x7ff0900ca870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff084001eb0 tx=0x7ff08400c860 comp rx=0 tx=0).stop 2023-07-06T20:15:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 shutdown_connections 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff07c06ad10 0x7ff07c06d1c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0900cfe80 0x7ff09014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0900b2180 0x7ff090156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 --2- 172.21.15.27:0/3382273589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0900b8610 0x7ff0900ca870 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 >> 172.21.15.27:0/3382273589 conn(0x7ff090071690 msgr2=0x7ff0900d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 shutdown_connections 2023-07-06T20:15:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.324+0000 7ff0963ff700 1 -- 172.21.15.27:0/3382273589 wait complete. 2023-07-06T20:15:12.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-06T20:15:12.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.548+0000 7f0e5bdff700 1 Processor -- start 2023-07-06T20:15:12.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.548+0000 7f0e5bdff700 1 -- start start 2023-07-06T20:15:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.549+0000 7f0e5bdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e5415fca0 0x7f0e540d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.549+0000 7f0e5bdff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e540b5c90 0x7f0e540d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.549+0000 7f0e5bdff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e540cf780 con 0x7f0e540b6110 2023-07-06T20:15:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.549+0000 7f0e5bdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e540cd490 con 0x7f0e5415fca0 2023-07-06T20:15:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.549+0000 7f0e5bdff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e540cd770 con 0x7f0e540b5c90 2023-07-06T20:15:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e5415fca0 0x7f0e540d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e5415fca0 0x7f0e540d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45442/0 (socket says 172.21.15.27:45442) 2023-07-06T20:15:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e5a39c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e540b6110 0x7f0e540cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40626/0 (socket says 172.21.15.27:40626) 2023-07-06T20:15:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 -- 172.21.15.27:0/525267219 learned_addr learned my addr 172.21.15.27:0/525267219 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e5939a700 1 --2- 172.21.15.27:0/525267219 >> v2:172.21.15.153:3300/0 conn(0x7f0e540b5c90 0x7f0e540d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 -- 172.21.15.27:0/525267219 >> v2:172.21.15.153:3300/0 conn(0x7f0e540b5c90 msgr2=0x7f0e540d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 --2- 172.21.15.27:0/525267219 >> v2:172.21.15.153:3300/0 conn(0x7f0e540b5c90 0x7f0e540d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 -- 172.21.15.27:0/525267219 >> v1:172.21.15.27:6789/0 conn(0x7f0e540b6110 legacy=0x7f0e540cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 -- 172.21.15.27:0/525267219 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540cda50 con 0x7f0e5415fca0 2023-07-06T20:15:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e5939a700 1 --2- 172.21.15.27:0/525267219 >> v2:172.21.15.153:3300/0 conn(0x7f0e540b5c90 0x7f0e540d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.550+0000 7f0e59b9b700 1 --2- 172.21.15.27:0/525267219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5415fca0 0x7f0e540d1420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e5406b6f0 tx=0x7f0e50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7885897d2d4a42 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e58b99700 1 -- 172.21.15.27:0/525267219 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000b870 con 0x7f0e5415fca0 2023-07-06T20:15:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e58b99700 1 -- 172.21.15.27:0/525267219 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5000b9d0 con 0x7f0e5415fca0 2023-07-06T20:15:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e58b99700 1 -- 172.21.15.27:0/525267219 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000bd60 con 0x7f0e5415fca0 2023-07-06T20:15:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 -- 172.21.15.27:0/525267219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5415fca0 msgr2=0x7f0e540d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/525267219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5415fca0 0x7f0e540d1420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e5406b6f0 tx=0x7f0e50009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 -- 172.21.15.27:0/525267219 shutdown_connections 2023-07-06T20:15:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/525267219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5415fca0 0x7f0e540d1420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/525267219 >> v2:172.21.15.153:3300/0 conn(0x7f0e540b5c90 0x7f0e540d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 -- 172.21.15.27:0/525267219 >> 172.21.15.27:0/525267219 conn(0x7f0e5406c150 msgr2=0x7f0e5406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 -- 172.21.15.27:0/525267219 shutdown_connections 2023-07-06T20:15:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.551+0000 7f0e5bdff700 1 -- 172.21.15.27:0/525267219 wait complete. 2023-07-06T20:15:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.553+0000 7f0e5bdff700 1 Processor -- start 2023-07-06T20:15:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.553+0000 7f0e5bdff700 1 -- start start 2023-07-06T20:15:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e5bdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 0x7f0e54148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e5bdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e540b6110 0x7f0e5415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e5bdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e5415fca0 0x7f0e5415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e5bdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5414edb0 con 0x7f0e540b5c90 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e5bdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5415f480 con 0x7f0e540b6110 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e5bdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5415e3e0 con 0x7f0e5415fca0 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e59b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 0x7f0e54148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e59b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 0x7f0e54148100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45452/0 (socket says 172.21.15.27:45452) 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.554+0000 7f0e59b9b700 1 -- 172.21.15.27:0/1588651360 learned_addr learned my addr 172.21.15.27:0/1588651360 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e59b9b700 1 -- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e5415fca0 msgr2=0x7f0e5415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e59b9b700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e5415fca0 0x7f0e5415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e5939a700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e540b6110 0x7f0e5415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e59b9b700 1 -- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e540b6110 msgr2=0x7f0e5415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e59b9b700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e540b6110 0x7f0e5415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e59b9b700 1 -- 172.21.15.27:0/1588651360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50009550 con 0x7f0e540b5c90 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e5939a700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e540b6110 0x7f0e5415b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e59b9b700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 0x7f0e54148100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0e50014040 tx=0x7f0e50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50004520 con 0x7f0e540b5c90 2023-07-06T20:15:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5001e070 con 0x7f0e540b5c90 2023-07-06T20:15:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5415e6c0 con 0x7f0e540b5c90 2023-07-06T20:15:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500038c0 con 0x7f0e540b5c90 2023-07-06T20:15:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.555+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5415ec80 con 0x7f0e540b5c90 2023-07-06T20:15:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.556+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e540008d0 con 0x7f0e540b5c90 2023-07-06T20:15:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.557+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e50003a20 con 0x7f0e540b5c90 2023-07-06T20:15:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.558+0000 7f0e4a7fc700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3806ace0 0x7f0e3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.558+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0e50020070 con 0x7f0e540b5c90 2023-07-06T20:15:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.559+0000 7f0e5939a700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3806ace0 0x7f0e3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.560+0000 7f0e5939a700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3806ace0 0x7f0e3806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e5415b590 tx=0x7f0e44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.563+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e50055c60 con 0x7f0e540b5c90 2023-07-06T20:15:12.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.849+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f0e540b2d40 con 0x7f0e540b5c90 2023-07-06T20:15:12.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.853+0000 7f0e4a7fc700 1 -- 172.21.15.27:0/1588651360 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0e50059280 con 0x7f0e540b5c90 2023-07-06T20:15:12.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3806ace0 msgr2=0x7f0e3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3806ace0 0x7f0e3806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e5415b590 tx=0x7f0e44008040 comp rx=0 tx=0).stop 2023-07-06T20:15:12.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 msgr2=0x7f0e54148100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:12.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 0x7f0e54148100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0e50014040 tx=0x7f0e50004750 comp rx=0 tx=0).stop 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 shutdown_connections 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3806ace0 0x7f0e3806d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e5415fca0 0x7f0e5415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e540b5c90 0x7f0e54148100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 --2- 172.21.15.27:0/1588651360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e540b6110 0x7f0e5415b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 >> 172.21.15.27:0/1588651360 conn(0x7f0e5406c150 msgr2=0x7f0e540cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 shutdown_connections 2023-07-06T20:15:12.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:12.855+0000 7f0e5bdff700 1 -- 172.21.15.27:0/1588651360 wait complete. 2023-07-06T20:15:12.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:13.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.077+0000 7fc131dff700 1 Processor -- start 2023-07-06T20:15:13.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.078+0000 7fc131dff700 1 -- start start 2023-07-06T20:15:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc131dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc131dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc12c0d0220 0x7fc12c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc131dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c06a8e0 con 0x7fc12c06a4a0 2023-07-06T20:15:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc131dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c0ca500 con 0x7fc12c0d0220 2023-07-06T20:15:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc131dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0d0690 con 0x7fc12c15c790 2023-07-06T20:15:13.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc123fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc12c0d0220 0x7fc12c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc12bfff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc12c06a4a0 0x7fc12c160340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40634/0 (socket says 172.21.15.27:40634) 2023-07-06T20:15:13.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.079+0000 7fc12bfff700 1 -- 172.21.15.27:0/2982747562 learned_addr learned my addr 172.21.15.27:0/2982747562 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:13.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc12b7fe700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc123fff700 1 -- 172.21.15.27:0/2982747562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 msgr2=0x7fc12c0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc123fff700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c0cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc123fff700 1 -- 172.21.15.27:0/2982747562 >> v1:172.21.15.27:6789/0 conn(0x7fc12c06a4a0 legacy=0x7fc12c160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc123fff700 1 -- 172.21.15.27:0/2982747562 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0ce6f0 con 0x7fc12c0d0220 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc12affd700 1 -- 172.21.15.27:0/2982747562 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222226461 0 0) 0x7fc12c06a8e0 con 0x7fc12c06a4a0 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc12b7fe700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c0cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc123fff700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c069f60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc114009840 tx=0x7fc114009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3612c62f3cd852ff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc12affd700 1 -- 172.21.15.27:0/2982747562 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114011830 con 0x7fc12c0d0220 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc12affd700 1 -- 172.21.15.27:0/2982747562 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114011990 con 0x7fc12c0d0220 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.080+0000 7fc12affd700 1 -- 172.21.15.27:0/2982747562 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114011d20 con 0x7fc12c0d0220 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 -- 172.21.15.27:0/2982747562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 msgr2=0x7fc12c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c069f60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc114009840 tx=0x7fc114009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 -- 172.21.15.27:0/2982747562 shutdown_connections 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c0cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 --2- 172.21.15.27:0/2982747562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c069f60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 -- 172.21.15.27:0/2982747562 >> 172.21.15.27:0/2982747562 conn(0x7fc12c0b22f0 msgr2=0x7fc12c0cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:13.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 -- 172.21.15.27:0/2982747562 shutdown_connections 2023-07-06T20:15:13.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.081+0000 7fc131dff700 1 -- 172.21.15.27:0/2982747562 wait complete. 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.083+0000 7fc131dff700 1 Processor -- start 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.084+0000 7fc131dff700 1 -- start start 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc131dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc12c06a4a0 0x7fc12c1f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc131dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc131dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c1e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc131dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0c75f0 con 0x7fc12c0d0220 2023-07-06T20:15:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc131dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0c9b20 con 0x7fc12c06a4a0 2023-07-06T20:15:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc131dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0c7840 con 0x7fc12c15c790 2023-07-06T20:15:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc123fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc12b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc12c06a4a0 0x7fc12c1f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc123fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c15b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45464/0 (socket says 172.21.15.27:45464) 2023-07-06T20:15:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc123fff700 1 -- 172.21.15.27:0/2266468484 learned_addr learned my addr 172.21.15.27:0/2266468484 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.085+0000 7fc12bfff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c1e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc123fff700 1 -- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 msgr2=0x7fc12c1e1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc123fff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c1e1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc123fff700 1 -- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc12c06a4a0 msgr2=0x7fc12c1f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc123fff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc12c06a4a0 0x7fc12c1f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc123fff700 1 -- 172.21.15.27:0/2266468484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140094a0 con 0x7fc12c0d0220 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc12b7fe700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc12c06a4a0 0x7fc12c1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc12bfff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c1e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc123fff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c15b310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0683d0 tx=0x7fc114004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.086+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140045d0 con 0x7fc12c0d0220 2023-07-06T20:15:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.087+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11401f070 con 0x7fc12c0d0220 2023-07-06T20:15:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.087+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140038e0 con 0x7fc12c0d0220 2023-07-06T20:15:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.087+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c0c7b20 con 0x7fc12c0d0220 2023-07-06T20:15:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.087+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c0c80e0 con 0x7fc12c0d0220 2023-07-06T20:15:13.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.088+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c0008d0 con 0x7fc12c0d0220 2023-07-06T20:15:13.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.089+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc114003a40 con 0x7fc12c0d0220 2023-07-06T20:15:13.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.089+0000 7fc128ff9700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc10c06ace0 0x7fc10c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.090+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc114021070 con 0x7fc12c0d0220 2023-07-06T20:15:13.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.090+0000 7fc12b7fe700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc10c06ace0 0x7fc10c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.091+0000 7fc12b7fe700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc10c06ace0 0x7fc10c06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc11c006fd0 tx=0x7fc11c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.094+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc114055f20 con 0x7fc12c0d0220 2023-07-06T20:15:13.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.303+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc12c0cf090 con 0x7fc12c0d0220 2023-07-06T20:15:13.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.304+0000 7fc128ff9700 1 -- 172.21.15.27:0/2266468484 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+14615 (secure 0 0 0) 0x7fc114059540 con 0x7fc12c0d0220 2023-07-06T20:15:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:13.306 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:09.095276+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc10c06ace0 msgr2=0x7fc10c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc10c06ace0 0x7fc10c06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc11c006fd0 tx=0x7fc11c008040 comp rx=0 tx=0).stop 2023-07-06T20:15:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 msgr2=0x7fc12c15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c15b310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0683d0 tx=0x7fc114004750 comp rx=0 tx=0).stop 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 shutdown_connections 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fc10c06ace0 0x7fc10c06d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc12c15c790 0x7fc12c1e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc12c06a4a0 0x7fc12c1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 --2- 172.21.15.27:0/2266468484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc12c0d0220 0x7fc12c15b310 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 >> 172.21.15.27:0/2266468484 conn(0x7fc12c0b22f0 msgr2=0x7fc12c0cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 shutdown_connections 2023-07-06T20:15:13.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.306+0000 7fc131dff700 1 -- 172.21.15.27:0/2266468484 wait complete. 2023-07-06T20:15:13.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:15:13.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:15:13.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.531+0000 7fef91dff700 1 Processor -- start 2023-07-06T20:15:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.532+0000 7fef91dff700 1 -- start start 2023-07-06T20:15:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.533+0000 7fef91dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.533+0000 7fef91dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0cf780 con 0x7fef8c0b60e0 2023-07-06T20:15:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.533+0000 7fef91dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0cd490 con 0x7fef8c15fca0 2023-07-06T20:15:13.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.533+0000 7fef91dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cd770 con 0x7fef8c0b5c90 2023-07-06T20:15:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.533+0000 7fef8bfff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fef8c0b60e0 0x7fef8c0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40636/0 (socket says 172.21.15.27:40636) 2023-07-06T20:15:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.533+0000 7fef8bfff700 1 -- 172.21.15.27:0/1929397962 learned_addr learned my addr 172.21.15.27:0/1929397962 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8affd700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589365481 0 0) 0x7fef8c0cf780 con 0x7fef8c0b60e0 2023-07-06T20:15:13.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef740035b0 con 0x7fef8c0b60e0 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299788876 0 0) 0x7fef8c0cd490 con 0x7fef8c15fca0 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef8c0cf780 con 0x7fef8c15fca0 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060654514 0 0) 0x7fef740035b0 con 0x7fef8c0b60e0 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 msgr2=0x7fef8c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 >> v1:172.21.15.153:6789/0 conn(0x7fef8c15fca0 legacy=0x7fef8c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0cda50 con 0x7fef8c0b60e0 2023-07-06T20:15:13.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7fef840019f0 con 0x7fef8c0b60e0 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 >> v1:172.21.15.27:6789/0 conn(0x7fef8c0b60e0 legacy=0x7fef8c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef74003bc0 0x7fef74003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74004d90 0x7fef74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cd770 con 0x7fef74004d90 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7400bac0 con 0x7fef74003bc0 2023-07-06T20:15:13.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.534+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7400bc50 con 0x7fef8c0b5c90 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8bfff700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74004d90 0x7fef74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8b7fe700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef74003bc0 0x7fef74003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 -- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef74003bc0 msgr2=0x7fef74003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef74003bc0 0x7fef74003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 -- 172.21.15.27:0/1929397962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74004d90 msgr2=0x7fef74005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74004d90 0x7fef74005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 -- 172.21.15.27:0/1929397962 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740035b0 con 0x7fef8c0b5c90 2023-07-06T20:15:13.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.535+0000 7fef8affd700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef74004850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef80009b60 tx=0x7fef74005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7545dbf125bfa733 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000e6c0 con 0x7fef8c0b5c90 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8000e820 con 0x7fef8c0b5c90 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef8a7fc700 1 -- 172.21.15.27:0/1929397962 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000eb90 con 0x7fef8c0b5c90 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef91dff700 1 -- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 msgr2=0x7fef74004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef91dff700 1 --2- 172.21.15.27:0/1929397962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef74004850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef80009b60 tx=0x7fef74005880 comp rx=0 tx=0).stop 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef8bfff700 1 -- 172.21.15.27:0/1929397962 reap_dead start 2023-07-06T20:15:13.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef91dff700 1 -- 172.21.15.27:0/1929397962 shutdown_connections 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef91dff700 1 -- 172.21.15.27:0/1929397962 >> 172.21.15.27:0/1929397962 conn(0x7fef8c06c150 msgr2=0x7fef8c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef91dff700 1 -- 172.21.15.27:0/1929397962 shutdown_connections 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.536+0000 7fef91dff700 1 -- 172.21.15.27:0/1929397962 wait complete. 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.538+0000 7fef91dff700 1 Processor -- start 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.538+0000 7fef91dff700 1 -- start start 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.539+0000 7fef91dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.539+0000 7fef91dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef8c1f1780 0x7fef8c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.539+0000 7fef91dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 0x7fef8c1f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.539+0000 7fef91dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1597e0 con 0x7fef8c159370 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.539+0000 7fef91dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cae30 con 0x7fef8c1f1780 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.539+0000 7fef91dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cb9d0 con 0x7fef8c0b5c90 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 0x7fef8c1f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 0x7fef8c1f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45486/0 (socket says 172.21.15.27:45486) 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 -- 172.21.15.27:0/827577162 learned_addr learned my addr 172.21.15.27:0/827577162 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:13.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8b7fe700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8affd700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef8c1f1780 0x7fef8c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 -- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 msgr2=0x7fef8c0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 -- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef8c1f1780 msgr2=0x7fef8c158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef8c1f1780 0x7fef8c158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 -- 172.21.15.27:0/827577162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0cbcb0 con 0x7fef8c159370 2023-07-06T20:15:13.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef8bfff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 0x7fef8c1f5470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0d3ed0 tx=0x7fef8400a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.540+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400ad30 con 0x7fef8c159370 2023-07-06T20:15:13.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.541+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef840133f0 con 0x7fef8c159370 2023-07-06T20:15:13.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.541+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c0cbf90 con 0x7fef8c159370 2023-07-06T20:15:13.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.541+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840137a0 con 0x7fef8c159370 2023-07-06T20:15:13.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.541+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c151760 con 0x7fef8c159370 2023-07-06T20:15:13.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.542+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c1512e0 con 0x7fef8c159370 2023-07-06T20:15:13.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.543+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef84013900 con 0x7fef8c159370 2023-07-06T20:15:13.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.544+0000 7fef73fff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fef7806ace0 0x7fef7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.544+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fef84089cb0 con 0x7fef8c159370 2023-07-06T20:15:13.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.544+0000 7fef8b7fe700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fef7806ace0 0x7fef7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.545+0000 7fef8b7fe700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fef7806ace0 0x7fef7806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef7c007fd0 tx=0x7fef7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.548+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef8c1512e0 con 0x7fef8c159370 2023-07-06T20:15:13.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.754+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef8c1512e0 con 0x7fef8c159370 2023-07-06T20:15:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.755+0000 7fef73fff700 1 -- 172.21.15.27:0/827577162 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef8c1512e0 con 0x7fef8c159370 2023-07-06T20:15:13.757 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:15:13.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fef7806ace0 msgr2=0x7fef7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fef7806ace0 0x7fef7806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef7c007fd0 tx=0x7fef7c009040 comp rx=0 tx=0).stop 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 msgr2=0x7fef8c1f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 0x7fef8c1f5470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0d3ed0 tx=0x7fef8400a5b0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 shutdown_connections 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fef7806ace0 0x7fef7806d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef8c159370 0x7fef8c1f5470 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef8c0b5c90 0x7fef8c0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 --2- 172.21.15.27:0/827577162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef8c1f1780 0x7fef8c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.757+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 >> 172.21.15.27:0/827577162 conn(0x7fef8c06c150 msgr2=0x7fef8c0d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.758+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 shutdown_connections 2023-07-06T20:15:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.758+0000 7fef91dff700 1 -- 172.21.15.27:0/827577162 wait complete. 2023-07-06T20:15:13.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:15:13.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.983+0000 7f79f63ff700 1 Processor -- start 2023-07-06T20:15:13.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.983+0000 7f79f63ff700 1 -- start start 2023-07-06T20:15:13.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.984+0000 7f79f63ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f79f015d8b0 0x7f79f00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.984+0000 7f79f63ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79f0160d50 0x7f79f00d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.984+0000 7f79f63ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f006cb60 con 0x7f79f015dc80 2023-07-06T20:15:13.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.984+0000 7f79f63ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f0161190 con 0x7f79f0160d50 2023-07-06T20:15:13.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.984+0000 7f79f63ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f01613a0 con 0x7f79f015d8b0 2023-07-06T20:15:13.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79f0160d50 0x7f79f00d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79f0160d50 0x7f79f00d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45498/0 (socket says 172.21.15.27:45498) 2023-07-06T20:15:13.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 -- 172.21.15.27:0/3821552578 learned_addr learned my addr 172.21.15.27:0/3821552578 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:13.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79effff700 1 --2- 172.21.15.27:0/3821552578 >> v2:172.21.15.153:3300/0 conn(0x7f79f015d8b0 0x7f79f00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 -- 172.21.15.27:0/3821552578 >> v2:172.21.15.153:3300/0 conn(0x7f79f015d8b0 msgr2=0x7f79f00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 --2- 172.21.15.27:0/3821552578 >> v2:172.21.15.153:3300/0 conn(0x7f79f015d8b0 0x7f79f00d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 -- 172.21.15.27:0/3821552578 >> v1:172.21.15.153:6789/0 conn(0x7f79f015dc80 legacy=0x7f79f0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 -- 172.21.15.27:0/3821552578 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f006c230 con 0x7f79f0160d50 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79effff700 1 --2- 172.21.15.27:0/3821552578 >> v2:172.21.15.153:3300/0 conn(0x7f79f015d8b0 0x7f79f00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.985+0000 7f79f499c700 1 --2- 172.21.15.27:0/3821552578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f0160d50 0x7f79f00d1080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009e50 tx=0x7f79e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd8fe96b70251df server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79eeffd700 1 -- 172.21.15.27:0/3821552578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800e070 con 0x7f79f0160d50 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79eeffd700 1 -- 172.21.15.27:0/3821552578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e800b980 con 0x7f79f0160d50 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79eeffd700 1 -- 172.21.15.27:0/3821552578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800bd30 con 0x7f79f0160d50 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 -- 172.21.15.27:0/3821552578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f0160d50 msgr2=0x7f79f00d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 --2- 172.21.15.27:0/3821552578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f0160d50 0x7f79f00d1080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009e50 tx=0x7f79e800b330 comp rx=0 tx=0).stop 2023-07-06T20:15:13.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 -- 172.21.15.27:0/3821552578 shutdown_connections 2023-07-06T20:15:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 --2- 172.21.15.27:0/3821552578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f0160d50 0x7f79f00d1080 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 --2- 172.21.15.27:0/3821552578 >> v2:172.21.15.153:3300/0 conn(0x7f79f015d8b0 0x7f79f00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 -- 172.21.15.27:0/3821552578 >> 172.21.15.27:0/3821552578 conn(0x7f79f006db20 msgr2=0x7f79f006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 -- 172.21.15.27:0/3821552578 shutdown_connections 2023-07-06T20:15:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.986+0000 7f79f63ff700 1 -- 172.21.15.27:0/3821552578 wait complete. 2023-07-06T20:15:13.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.988+0000 7f79f63ff700 1 Processor -- start 2023-07-06T20:15:13.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.988+0000 7f79f63ff700 1 -- start start 2023-07-06T20:15:13.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79f63ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 0x7f79f00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79f63ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79f0160d50 0x7f79f0147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79f63ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f00c6d30 0x7f79f00c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79f63ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b0ea0 con 0x7f79f015d8b0 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79f63ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00affb0 con 0x7f79f00c6d30 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79f63ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b0290 con 0x7f79f0160d50 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 0x7f79f00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 0x7f79f00b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45506/0 (socket says 172.21.15.27:45506) 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.989+0000 7f79effff700 1 -- 172.21.15.27:0/2323872674 learned_addr learned my addr 172.21.15.27:0/2323872674 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79ef7fe700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79f0160d50 0x7f79f0147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79f499c700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f00c6d30 0x7f79f00c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79effff700 1 -- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79f0160d50 msgr2=0x7f79f0147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79effff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79f0160d50 0x7f79f0147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79effff700 1 -- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f00c6d30 msgr2=0x7f79f00c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79effff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f00c6d30 0x7f79f00c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79effff700 1 -- 172.21.15.27:0/2323872674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8009b30 con 0x7f79f015d8b0 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79f499c700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f00c6d30 0x7f79f00c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79effff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 0x7f79f00b1330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e810 tx=0x7f79e000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0004230 con 0x7f79f015d8b0 2023-07-06T20:15:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0004390 con 0x7f79f015d8b0 2023-07-06T20:15:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e00036a0 con 0x7f79f015d8b0 2023-07-06T20:15:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.990+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f00b0570 con 0x7f79f015d8b0 2023-07-06T20:15:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.991+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f00b0b30 con 0x7f79f015d8b0 2023-07-06T20:15:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.992+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f006a270 con 0x7f79f015d8b0 2023-07-06T20:15:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.993+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79e0003800 con 0x7f79f015d8b0 2023-07-06T20:15:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.994+0000 7f79ecff9700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f79dc06ace0 0x7f79dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.994+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f79e0088c70 con 0x7f79f015d8b0 2023-07-06T20:15:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.995+0000 7f79ef7fe700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f79dc06ace0 0x7f79dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.995+0000 7f79ef7fe700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f79dc06ace0 0x7f79dc06d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f79f01572f0 tx=0x7f79e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:13.998+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79e0054070 con 0x7f79f015d8b0 2023-07-06T20:15:14.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.207+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79f00008d0 con 0x7f79f015d8b0 2023-07-06T20:15:14.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.207+0000 7f79ecff9700 1 -- 172.21.15.27:0/2323872674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79e0057690 con 0x7f79f015d8b0 2023-07-06T20:15:14.209 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:15:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.209+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f79dc06ace0 msgr2=0x7f79dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.209+0000 7f79f63ff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f79dc06ace0 0x7f79dc06d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f79f01572f0 tx=0x7f79e4009040 comp rx=0 tx=0).stop 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.209+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 msgr2=0x7f79f00b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.209+0000 7f79f63ff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 0x7f79f00b1330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e810 tx=0x7f79e000ebd0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 shutdown_connections 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f79dc06ace0 0x7f79dc06d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f00c6d30 0x7f79f00c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f015d8b0 0x7f79f00b1330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 --2- 172.21.15.27:0/2323872674 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f79f0160d50 0x7f79f0147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 >> 172.21.15.27:0/2323872674 conn(0x7f79f006db20 msgr2=0x7f79f0152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 shutdown_connections 2023-07-06T20:15:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.210+0000 7f79f63ff700 1 -- 172.21.15.27:0/2323872674 wait complete. 2023-07-06T20:15:14.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T20:15:14.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.432+0000 7f53e3bff700 1 Processor -- start 2023-07-06T20:15:14.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.432+0000 7f53e3bff700 1 -- start start 2023-07-06T20:15:14.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e3bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53dc0b5c90 0x7f53dc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e3bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc0cf780 con 0x7f53dc0b6110 2023-07-06T20:15:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e3bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc0cd490 con 0x7f53dc15fca0 2023-07-06T20:15:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e3bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53dc0cd770 con 0x7f53dc0b5c90 2023-07-06T20:15:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e119a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53dc0b5c90 0x7f53dc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e219c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f53dc0b6110 0x7f53dc0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40652/0 (socket says 172.21.15.27:40652) 2023-07-06T20:15:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e219c700 1 -- 172.21.15.27:0/227176718 learned_addr learned my addr 172.21.15.27:0/227176718 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e119a700 1 -- 172.21.15.27:0/227176718 >> v1:172.21.15.153:6789/0 conn(0x7f53dc15fca0 legacy=0x7f53dc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e119a700 1 -- 172.21.15.27:0/227176718 >> v1:172.21.15.27:6789/0 conn(0x7f53dc0b6110 legacy=0x7f53dc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.434+0000 7f53e119a700 1 -- 172.21.15.27:0/227176718 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0cda50 con 0x7f53dc0b5c90 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.435+0000 7f53e119a700 1 --2- 172.21.15.27:0/227176718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc0b5c90 0x7f53dc0d2d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009840 tx=0x7f53cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6279eb905aca74e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.435+0000 7f53e0999700 1 -- 172.21.15.27:0/227176718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc011830 con 0x7f53dc0b5c90 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.435+0000 7f53e0999700 1 -- 172.21.15.27:0/227176718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc011990 con 0x7f53dc0b5c90 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.435+0000 7f53e0999700 1 -- 172.21.15.27:0/227176718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc011d20 con 0x7f53dc0b5c90 2023-07-06T20:15:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.435+0000 7f53e3bff700 1 -- 172.21.15.27:0/227176718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc0b5c90 msgr2=0x7f53dc0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.435+0000 7f53e3bff700 1 --2- 172.21.15.27:0/227176718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc0b5c90 0x7f53dc0d2d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009840 tx=0x7f53cc009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.436+0000 7f53e3bff700 1 -- 172.21.15.27:0/227176718 shutdown_connections 2023-07-06T20:15:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.436+0000 7f53e3bff700 1 --2- 172.21.15.27:0/227176718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc0b5c90 0x7f53dc0d2d40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.436+0000 7f53e3bff700 1 -- 172.21.15.27:0/227176718 >> 172.21.15.27:0/227176718 conn(0x7f53dc06c150 msgr2=0x7f53dc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.436+0000 7f53e3bff700 1 -- 172.21.15.27:0/227176718 shutdown_connections 2023-07-06T20:15:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.436+0000 7f53e3bff700 1 -- 172.21.15.27:0/227176718 wait complete. 2023-07-06T20:15:14.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.437+0000 7f53e3bff700 1 Processor -- start 2023-07-06T20:15:14.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 -- start start 2023-07-06T20:15:14.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53dc0b5c90 0x7f53dc151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53dc0b6110 0x7f53dc0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 0x7f53dc0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc1575b0 con 0x7f53dc15fca0 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0c8ae0 con 0x7f53dc0b5c90 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.438+0000 7f53e3bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0c7ed0 con 0x7f53dc0b6110 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 0x7f53dc0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 0x7f53dc0cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45516/0 (socket says 172.21.15.27:45516) 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 -- 172.21.15.27:0/1382006631 learned_addr learned my addr 172.21.15.27:0/1382006631 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e119a700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53dc0b6110 0x7f53dc0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e199b700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53dc0b5c90 0x7f53dc151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 -- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53dc0b6110 msgr2=0x7f53dc0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53dc0b6110 0x7f53dc0cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 -- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53dc0b5c90 msgr2=0x7f53dc151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53dc0b5c90 0x7f53dc151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 -- 172.21.15.27:0/1382006631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0094a0 con 0x7f53dc15fca0 2023-07-06T20:15:14.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.439+0000 7f53e219c700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 0x7f53dc0cbca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f53d400cbc0 tx=0x7f53d400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.440+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400ab50 con 0x7f53dc15fca0 2023-07-06T20:15:14.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.440+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400acb0 con 0x7f53dc15fca0 2023-07-06T20:15:14.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.440+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4012600 con 0x7f53dc15fca0 2023-07-06T20:15:14.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.440+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc0c81b0 con 0x7f53dc15fca0 2023-07-06T20:15:14.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.440+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc0c8770 con 0x7f53dc15fca0 2023-07-06T20:15:14.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.441+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53dc0c8c50 con 0x7f53dc15fca0 2023-07-06T20:15:14.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.442+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53d401b3b0 con 0x7f53dc15fca0 2023-07-06T20:15:14.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.443+0000 7f53d27fc700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53c006ace0 0x7f53c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.443+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f53d4088e80 con 0x7f53dc15fca0 2023-07-06T20:15:14.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.443+0000 7f53e199b700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53c006ace0 0x7f53c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.444+0000 7f53e199b700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53c006ace0 0x7f53c006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f60 tx=0x7f53d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.447+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53d4054200 con 0x7f53dc15fca0 2023-07-06T20:15:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.659+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f53dc0008d0 con 0x7f53dc15fca0 2023-07-06T20:15:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.660+0000 7f53d27fc700 1 -- 172.21.15.27:0/1382006631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+126 (secure 0 0 0) 0x7f53d4017020 con 0x7f53dc15fca0 2023-07-06T20:15:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:14.662 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-07-06T20:15:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-07-06T20:15:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-07-06T20:15:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-07-06T20:15:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-07-06T20:15:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 2, 2023-07-06T20:15:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": "rbd" 2023-07-06T20:15:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-07-06T20:15:14.663 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-07-06T20:15:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53c006ace0 msgr2=0x7f53c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53c006ace0 0x7f53c006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f60 tx=0x7f53d8009040 comp rx=0 tx=0).stop 2023-07-06T20:15:14.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 msgr2=0x7f53dc0cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 0x7f53dc0cbca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f53d400cbc0 tx=0x7f53d400cf80 comp rx=0 tx=0).stop 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 shutdown_connections 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53c006ace0 0x7f53c006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53dc15fca0 0x7f53dc0cbca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53dc0b5c90 0x7f53dc151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 --2- 172.21.15.27:0/1382006631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53dc0b6110 0x7f53dc0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 >> 172.21.15.27:0/1382006631 conn(0x7f53dc06c150 msgr2=0x7f53dc0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:14.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.662+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 shutdown_connections 2023-07-06T20:15:14.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.663+0000 7f53e3bff700 1 -- 172.21.15.27:0/1382006631 wait complete. 2023-07-06T20:15:14.682 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T20:15:14.682 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T20:15:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.888+0000 7f2e223ff700 1 Processor -- start 2023-07-06T20:15:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.888+0000 7f2e223ff700 1 -- start start 2023-07-06T20:15:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.889+0000 7f2e223ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2e1c0b8440 0x7f2e1c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.889+0000 7f2e223ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0cfe40 0x7f2e1c0d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.889+0000 7f2e223ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c160cc0 con 0x7f2e1c0b2180 2023-07-06T20:15:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.889+0000 7f2e223ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c160ae0 con 0x7f2e1c0b8440 2023-07-06T20:15:14.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.889+0000 7f2e223ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c160f80 con 0x7f2e1c0cfe40 2023-07-06T20:15:14.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2e1c0b8440 0x7f2e1c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2e1c0b8440 0x7f2e1c0cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45526/0 (socket says 172.21.15.27:45526) 2023-07-06T20:15:14.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 -- 172.21.15.27:0/1545681437 learned_addr learned my addr 172.21.15.27:0/1545681437 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:14.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e2099c700 1 --2- 172.21.15.27:0/1545681437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0cfe40 0x7f2e1c0d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 -- 172.21.15.27:0/1545681437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0cfe40 msgr2=0x7f2e1c0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 --2- 172.21.15.27:0/1545681437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0cfe40 0x7f2e1c0d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 -- 172.21.15.27:0/1545681437 >> v1:172.21.15.153:6789/0 conn(0x7f2e1c0b2180 legacy=0x7f2e1c0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 -- 172.21.15.27:0/1545681437 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0cf0e0 con 0x7f2e1c0b8440 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.890+0000 7f2e1bfff700 1 --2- 172.21.15.27:0/1545681437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c0cd530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0098f0 tx=0x7f2e0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8421a391e86f0fa2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e1affd700 1 -- 172.21.15.27:0/1545681437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00b8a0 con 0x7f2e1c0b8440 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e1affd700 1 -- 172.21.15.27:0/1545681437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00ba00 con 0x7f2e1c0b8440 2023-07-06T20:15:14.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e1affd700 1 -- 172.21.15.27:0/1545681437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00bd90 con 0x7f2e1c0b8440 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 -- 172.21.15.27:0/1545681437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 msgr2=0x7f2e1c0cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 --2- 172.21.15.27:0/1545681437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c0cd530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0098f0 tx=0x7f2e0c009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 -- 172.21.15.27:0/1545681437 shutdown_connections 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 --2- 172.21.15.27:0/1545681437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0cfe40 0x7f2e1c0d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 --2- 172.21.15.27:0/1545681437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c0cd530 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 -- 172.21.15.27:0/1545681437 >> 172.21.15.27:0/1545681437 conn(0x7f2e1c06be10 msgr2=0x7f2e1c071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:14.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 -- 172.21.15.27:0/1545681437 shutdown_connections 2023-07-06T20:15:14.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.891+0000 7f2e223ff700 1 -- 172.21.15.27:0/1545681437 wait complete. 2023-07-06T20:15:14.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.894+0000 7f2e223ff700 1 Processor -- start 2023-07-06T20:15:14.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.894+0000 7f2e223ff700 1 -- start start 2023-07-06T20:15:14.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.895+0000 7f2e223ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0b2180 0x7f2e1c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.895+0000 7f2e223ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.895+0000 7f2e223ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e1c0cfe40 0x7f2e1c1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.895+0000 7f2e223ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c8630 con 0x7f2e1c0b8440 2023-07-06T20:15:14.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.895+0000 7f2e223ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c88f0 con 0x7f2e1c0cfe40 2023-07-06T20:15:14.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.895+0000 7f2e223ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c8bd0 con 0x7f2e1c0b2180 2023-07-06T20:15:14.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45538/0 (socket says 172.21.15.27:45538) 2023-07-06T20:15:14.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 -- 172.21.15.27:0/3563134361 learned_addr learned my addr 172.21.15.27:0/3563134361 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:14.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1bfff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0b2180 0x7f2e1c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e2099c700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e1c0cfe40 0x7f2e1c1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 -- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0b2180 msgr2=0x7f2e1c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0b2180 0x7f2e1c0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 -- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e1c0cfe40 msgr2=0x7f2e1c1485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e1c0cfe40 0x7f2e1c1485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 -- 172.21.15.27:0/3563134361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009550 con 0x7f2e1c0b8440 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.896+0000 7f2e1b7fe700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c148080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c4b0 tx=0x7f2e1000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.897+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e100052d0 con 0x7f2e1c0b8440 2023-07-06T20:15:14.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.897+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e10005430 con 0x7f2e1c0b8440 2023-07-06T20:15:14.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.897+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e100038c0 con 0x7f2e1c0b8440 2023-07-06T20:15:14.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.897+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c0c8eb0 con 0x7f2e1c0b8440 2023-07-06T20:15:14.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.897+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c0c9470 con 0x7f2e1c0b8440 2023-07-06T20:15:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.898+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c0c9630 con 0x7f2e1c0b8440 2023-07-06T20:15:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.899+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e10003a20 con 0x7f2e1c0b8440 2023-07-06T20:15:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.900+0000 7f2e18ff9700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2e0806ace0 0x7f2e0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.900+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2e10088460 con 0x7f2e1c0b8440 2023-07-06T20:15:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.900+0000 7f2e1bfff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2e0806ace0 0x7f2e0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:14.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.901+0000 7f2e1bfff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2e0806ace0 0x7f2e0806d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009c40 tx=0x7f2e0c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:14.904+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2e10053860 con 0x7f2e1c0b8440 2023-07-06T20:15:15.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.110+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2e1c0008d0 con 0x7f2e1c0b8440 2023-07-06T20:15:15.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.197+0000 7f2e18ff9700 1 -- 172.21.15.27:0/3563134361 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2e10056e80 con 0x7f2e1c0b8440 2023-07-06T20:15:15.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2e0806ace0 msgr2=0x7f2e0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:15.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2e0806ace0 0x7f2e0806d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009c40 tx=0x7f2e0c008030 comp rx=0 tx=0).stop 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 msgr2=0x7f2e1c148080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c148080 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c4b0 tx=0x7f2e1000c7c0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 shutdown_connections 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f2e0806ace0 0x7f2e0806d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e1c0cfe40 0x7f2e1c1485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2e1c0b2180 0x7f2e1c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 --2- 172.21.15.27:0/3563134361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c0b8440 0x7f2e1c148080 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 >> 172.21.15.27:0/3563134361 conn(0x7f2e1c06be10 msgr2=0x7f2e1c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.200+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 shutdown_connections 2023-07-06T20:15:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.201+0000 7f2e223ff700 1 -- 172.21.15.27:0/3563134361 wait complete. 2023-07-06T20:15:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-07-06T20:15:15.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T20:15:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.431+0000 7f73639ff700 1 Processor -- start 2023-07-06T20:15:15.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.431+0000 7f73639ff700 1 -- start start 2023-07-06T20:15:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.432+0000 7f73639ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f735c15fca0 0x7f735c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.432+0000 7f73639ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f735c0b5c90 0x7f735c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.432+0000 7f73639ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c0cf780 con 0x7f735c0b6110 2023-07-06T20:15:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.432+0000 7f73639ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c0cd490 con 0x7f735c15fca0 2023-07-06T20:15:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.432+0000 7f73639ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c0cd770 con 0x7f735c0b5c90 2023-07-06T20:15:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.432+0000 7f736179b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f735c15fca0 0x7f735c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f735c15fca0 0x7f735c0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45548/0 (socket says 172.21.15.27:45548) 2023-07-06T20:15:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 -- 172.21.15.27:0/4093892125 learned_addr learned my addr 172.21.15.27:0/4093892125 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f7360f9a700 1 --2- 172.21.15.27:0/4093892125 >> v2:172.21.15.153:3300/0 conn(0x7f735c0b5c90 0x7f735c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 -- 172.21.15.27:0/4093892125 >> v2:172.21.15.153:3300/0 conn(0x7f735c0b5c90 msgr2=0x7f735c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 --2- 172.21.15.27:0/4093892125 >> v2:172.21.15.153:3300/0 conn(0x7f735c0b5c90 0x7f735c0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 -- 172.21.15.27:0/4093892125 >> v1:172.21.15.153:6789/0 conn(0x7f735c0b6110 legacy=0x7f735c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 -- 172.21.15.27:0/4093892125 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0cda50 con 0x7f735c15fca0 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.433+0000 7f736179b700 1 --2- 172.21.15.27:0/4093892125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d1420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f735c06b6f0 tx=0x7f7358009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141bcfa60b792ec server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f7353fff700 1 -- 172.21.15.27:0/4093892125 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358011830 con 0x7f735c15fca0 2023-07-06T20:15:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f7353fff700 1 -- 172.21.15.27:0/4093892125 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358011990 con 0x7f735c15fca0 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f7353fff700 1 -- 172.21.15.27:0/4093892125 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358011d20 con 0x7f735c15fca0 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 -- 172.21.15.27:0/4093892125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 msgr2=0x7f735c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 --2- 172.21.15.27:0/4093892125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d1420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f735c06b6f0 tx=0x7f7358009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 -- 172.21.15.27:0/4093892125 shutdown_connections 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 --2- 172.21.15.27:0/4093892125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d1420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 --2- 172.21.15.27:0/4093892125 >> v2:172.21.15.153:3300/0 conn(0x7f735c0b5c90 0x7f735c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 -- 172.21.15.27:0/4093892125 >> 172.21.15.27:0/4093892125 conn(0x7f735c06c150 msgr2=0x7f735c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 -- 172.21.15.27:0/4093892125 shutdown_connections 2023-07-06T20:15:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.434+0000 7f73639ff700 1 -- 172.21.15.27:0/4093892125 wait complete. 2023-07-06T20:15:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.436+0000 7f73639ff700 1 Processor -- start 2023-07-06T20:15:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.436+0000 7f73639ff700 1 -- start start 2023-07-06T20:15:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f73639ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f735c0b5c90 0x7f735c1e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f73639ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f735c0b6110 0x7f735c0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f73639ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f73639ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0c8d80 con 0x7f735c15fca0 2023-07-06T20:15:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f73639ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c1e6eb0 con 0x7f735c0b5c90 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f73639ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0c7ea0 con 0x7f735c0b6110 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f7361f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f7361f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45550/0 (socket says 172.21.15.27:45550) 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.437+0000 7f7361f9c700 1 -- 172.21.15.27:0/3253050648 learned_addr learned my addr 172.21.15.27:0/3253050648 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f736179b700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f735c0b5c90 0x7f735c1e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7360f9a700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f735c0b6110 0x7f735c0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7361f9c700 1 -- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f735c0b6110 msgr2=0x7f735c0c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7361f9c700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f735c0b6110 0x7f735c0c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7361f9c700 1 -- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f735c0b5c90 msgr2=0x7f735c1e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7361f9c700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f735c0b5c90 0x7f735c1e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7361f9c700 1 -- 172.21.15.27:0/3253050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358009550 con 0x7f735c15fca0 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7361f9c700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d2d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f735400cc50 tx=0x7f735400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354003c00 con 0x7f735c15fca0 2023-07-06T20:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354003d60 con 0x7f735c15fca0 2023-07-06T20:15:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c0c8100 con 0x7f735c15fca0 2023-07-06T20:15:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.438+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735400ab60 con 0x7f735c15fca0 2023-07-06T20:15:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.439+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c0c86c0 con 0x7f735c15fca0 2023-07-06T20:15:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.440+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c0008d0 con 0x7f735c15fca0 2023-07-06T20:15:15.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.442+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f735400acc0 con 0x7f735c15fca0 2023-07-06T20:15:15.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.443+0000 7f7351ffb700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f734006ace0 0x7f734006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:15.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.443+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f7354089d90 con 0x7f735c15fca0 2023-07-06T20:15:15.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.444+0000 7f736179b700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f734006ace0 0x7f734006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.444+0000 7f736179b700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f734006ace0 0x7f734006d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7358014040 tx=0x7f7358008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.447+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7354055090 con 0x7f735c15fca0 2023-07-06T20:15:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:15.654+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f735c15a640 con 0x7f735c15fca0 2023-07-06T20:15:16.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.197+0000 7f7351ffb700 1 -- 172.21.15.27:0/3253050648 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f73540586b0 con 0x7f735c15fca0 2023-07-06T20:15:16.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.206+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f734006ace0 msgr2=0x7f734006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:16.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.206+0000 7f73639ff700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f734006ace0 0x7f734006d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7358014040 tx=0x7f7358008030 comp rx=0 tx=0).stop 2023-07-06T20:15:16.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.206+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 msgr2=0x7f735c0d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:16.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.206+0000 7f73639ff700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d2d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f735400cc50 tx=0x7f735400cf60 comp rx=0 tx=0).stop 2023-07-06T20:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.210+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 shutdown_connections 2023-07-06T20:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.210+0000 7f73639ff700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f734006ace0 0x7f734006d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.210+0000 7f73639ff700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735c15fca0 0x7f735c0d2d70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.210+0000 7f73639ff700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f735c0b5c90 0x7f735c1e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.210+0000 7f73639ff700 1 --2- 172.21.15.27:0/3253050648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f735c0b6110 0x7f735c0c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.210+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 >> 172.21.15.27:0/3253050648 conn(0x7f735c06c150 msgr2=0x7f735c0b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.211+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 shutdown_connections 2023-07-06T20:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.211+0000 7f73639ff700 1 -- 172.21.15.27:0/3253050648 wait complete. 2023-07-06T20:15:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-07-06T20:15:16.238 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T20:15:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.444+0000 7fe7a4bff700 1 Processor -- start 2023-07-06T20:15:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.444+0000 7fe7a4bff700 1 -- start start 2023-07-06T20:15:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.445+0000 7fe7a4bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7a00b5c90 0x7fe7a00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.445+0000 7fe7a4bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe7a00b6110 0x7fe7a00d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.445+0000 7fe7a4bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00cf780 con 0x7fe7a015fca0 2023-07-06T20:15:16.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.445+0000 7fe7a4bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00cd490 con 0x7fe7a00b5c90 2023-07-06T20:15:16.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.445+0000 7fe7a4bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00cd770 con 0x7fe7a00b6110 2023-07-06T20:15:16.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7a00b5c90 0x7fe7a00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:16.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7a00b5c90 0x7fe7a00d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45552/0 (socket says 172.21.15.27:45552) 2023-07-06T20:15:16.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 -- 172.21.15.27:0/173840112 learned_addr learned my addr 172.21.15.27:0/173840112 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:16.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 -- 172.21.15.27:0/173840112 >> v2:172.21.15.153:3300/0 conn(0x7fe7a00b6110 msgr2=0x7fe7a00d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79ed9d700 1 --2- 172.21.15.27:0/173840112 >> v2:172.21.15.153:3300/0 conn(0x7fe7a00b6110 0x7fe7a00d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 --2- 172.21.15.27:0/173840112 >> v2:172.21.15.153:3300/0 conn(0x7fe7a00b6110 0x7fe7a00d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 -- 172.21.15.27:0/173840112 >> v1:172.21.15.153:6789/0 conn(0x7fe7a015fca0 legacy=0x7fe7a00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 -- 172.21.15.27:0/173840112 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00cda50 con 0x7fe7a00b5c90 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79ed9d700 1 --2- 172.21.15.27:0/173840112 >> v2:172.21.15.153:3300/0 conn(0x7fe7a00b6110 0x7fe7a00d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.446+0000 7fe79dd9b700 1 --2- 172.21.15.27:0/173840112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a00d2d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00cd410 tx=0x7fe794009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e44be17ed26a23b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe79d59a700 1 -- 172.21.15.27:0/173840112 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794004230 con 0x7fe7a00b5c90 2023-07-06T20:15:16.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe79d59a700 1 -- 172.21.15.27:0/173840112 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe794004390 con 0x7fe7a00b5c90 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe79d59a700 1 -- 172.21.15.27:0/173840112 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940036a0 con 0x7fe7a00b5c90 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 -- 172.21.15.27:0/173840112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 msgr2=0x7fe7a00d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/173840112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a00d2d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00cd410 tx=0x7fe794009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 -- 172.21.15.27:0/173840112 shutdown_connections 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/173840112 >> v2:172.21.15.153:3300/0 conn(0x7fe7a00b6110 0x7fe7a00d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/173840112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a00d2d40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 -- 172.21.15.27:0/173840112 >> 172.21.15.27:0/173840112 conn(0x7fe7a006c150 msgr2=0x7fe7a006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:16.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 -- 172.21.15.27:0/173840112 shutdown_connections 2023-07-06T20:15:16.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.447+0000 7fe7a4bff700 1 -- 172.21.15.27:0/173840112 wait complete. 2023-07-06T20:15:16.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.449+0000 7fe7a4bff700 1 Processor -- start 2023-07-06T20:15:16.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.449+0000 7fe7a4bff700 1 -- start start 2023-07-06T20:15:16.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe7a4bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a0157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:16.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe7a4bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7a00b6110 0x7fe7a00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:16.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe7a4bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7a015fca0 0x7fe7a00d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:16.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe7a4bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00c7770 con 0x7fe7a00b5c90 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe7a4bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00c7440 con 0x7fe7a00b6110 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe7a4bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00cb550 con 0x7fe7a015fca0 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.450+0000 7fe79e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a0157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45566/0 (socket says 172.21.15.27:45566) 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 -- 172.21.15.27:0/1082548298 learned_addr learned my addr 172.21.15.27:0/1082548298 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79dd9b700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7a00b6110 0x7fe7a00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79ed9d700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7a015fca0 0x7fe7a00d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:16.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 -- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7a015fca0 msgr2=0x7fe7a00d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:16.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7a015fca0 0x7fe7a00d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 -- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7a00b6110 msgr2=0x7fe7a00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:16.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7a00b6110 0x7fe7a00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:16.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 -- 172.21.15.27:0/1082548298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009550 con 0x7fe7a00b5c90 2023-07-06T20:15:16.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79ed9d700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7a015fca0 0x7fe7a00d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:16.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe79e59c700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a0157540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b4a0 tx=0x7fe79000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:16.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.451+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790011830 con 0x7fe7a00b5c90 2023-07-06T20:15:16.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.452+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790011990 con 0x7fe7a00b5c90 2023-07-06T20:15:16.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.452+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790011d40 con 0x7fe7a00b5c90 2023-07-06T20:15:16.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.452+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a0157a80 con 0x7fe7a00b5c90 2023-07-06T20:15:16.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.452+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00caf90 con 0x7fe7a00b5c90 2023-07-06T20:15:16.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.453+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a00008d0 con 0x7fe7a00b5c90 2023-07-06T20:15:16.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.454+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe79001a400 con 0x7fe7a00b5c90 2023-07-06T20:15:16.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.455+0000 7fe78effd700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe78006ace0 0x7fe78006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:16.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.455+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe790088ff0 con 0x7fe7a00b5c90 2023-07-06T20:15:16.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.456+0000 7fe79dd9b700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe78006ace0 0x7fe78006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:16.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.456+0000 7fe79dd9b700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe78006ace0 0x7fe78006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe794009130 tx=0x7fe794008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:16.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.460+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe790054190 con 0x7fe7a00b5c90 2023-07-06T20:15:16.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:16.712+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe7a00738f0 con 0x7fe7a00b5c90 2023-07-06T20:15:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.233+0000 7fe78effd700 1 -- 172.21.15.27:0/1082548298 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe7900577b0 con 0x7fe7a00b5c90 2023-07-06T20:15:17.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.247+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe78006ace0 msgr2=0x7fe78006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.247+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe78006ace0 0x7fe78006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe794009130 tx=0x7fe794008030 comp rx=0 tx=0).stop 2023-07-06T20:15:17.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.247+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 msgr2=0x7fe7a0157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.247+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a0157540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b4a0 tx=0x7fe79000b7b0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 shutdown_connections 2023-07-06T20:15:17.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe78006ace0 0x7fe78006d190 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe794009130 tx=0x7fe794008030 comp rx=0 tx=0).stop 2023-07-06T20:15:17.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe7a015fca0 0x7fe7a00d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00b5c90 0x7fe7a0157540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 --2- 172.21.15.27:0/1082548298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7a00b6110 0x7fe7a00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 >> 172.21.15.27:0/1082548298 conn(0x7fe7a006c150 msgr2=0x7fe7a00b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:17.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.248+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 shutdown_connections 2023-07-06T20:15:17.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.249+0000 7fe7a4bff700 1 -- 172.21.15.27:0/1082548298 wait complete. 2023-07-06T20:15:17.252 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 5 and data pool 6 2023-07-06T20:15:17.272 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:17.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.473+0000 7f969b1ff700 1 Processor -- start 2023-07-06T20:15:17.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.473+0000 7f969b1ff700 1 -- start start 2023-07-06T20:15:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.474+0000 7f969b1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f969415fca0 0x7f96940d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.474+0000 7f969b1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b6110 0x7f96940cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.474+0000 7f969b1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96940cf780 con 0x7f96940b5c90 2023-07-06T20:15:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.474+0000 7f969b1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96940cd490 con 0x7f969415fca0 2023-07-06T20:15:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f969b1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96940cd770 con 0x7f96940b6110 2023-07-06T20:15:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f969979c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b6110 0x7f96940cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f9693fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f96940b5c90 0x7f96940d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:37352/0 (socket says 172.21.15.27:37352) 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f9693fff700 1 -- 172.21.15.27:0/1527520129 learned_addr learned my addr 172.21.15.27:0/1527520129 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f9698f9b700 1 --2- 172.21.15.27:0/1527520129 >> v2:172.21.15.153:3300/0 conn(0x7f969415fca0 0x7f96940d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f969979c700 1 -- 172.21.15.27:0/1527520129 >> v2:172.21.15.153:3300/0 conn(0x7f969415fca0 msgr2=0x7f96940d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f969979c700 1 --2- 172.21.15.27:0/1527520129 >> v2:172.21.15.153:3300/0 conn(0x7f969415fca0 0x7f96940d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f969979c700 1 -- 172.21.15.27:0/1527520129 >> v1:172.21.15.153:6789/0 conn(0x7f96940b5c90 legacy=0x7f96940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f96937fe700 1 -- 172.21.15.27:0/1527520129 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155561079 0 0) 0x7f96940cf780 con 0x7f96940b5c90 2023-07-06T20:15:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.475+0000 7f969979c700 1 -- 172.21.15.27:0/1527520129 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940cda50 con 0x7f96940b6110 2023-07-06T20:15:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.476+0000 7f969979c700 1 --2- 172.21.15.27:0/1527520129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b6110 0x7f96940cfc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f968c009d90 tx=0x7f968c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24fdec81a16d4b2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.476+0000 7f96937fe700 1 -- 172.21.15.27:0/1527520129 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00e070 con 0x7f96940b6110 2023-07-06T20:15:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.476+0000 7f96937fe700 1 -- 172.21.15.27:0/1527520129 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c00bb20 con 0x7f96940b6110 2023-07-06T20:15:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.476+0000 7f96937fe700 1 -- 172.21.15.27:0/1527520129 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c012430 con 0x7f96940b6110 2023-07-06T20:15:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 -- 172.21.15.27:0/1527520129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b6110 msgr2=0x7f96940cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 --2- 172.21.15.27:0/1527520129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b6110 0x7f96940cfc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f968c009d90 tx=0x7f968c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 -- 172.21.15.27:0/1527520129 shutdown_connections 2023-07-06T20:15:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 --2- 172.21.15.27:0/1527520129 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b6110 0x7f96940cfc10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 --2- 172.21.15.27:0/1527520129 >> v2:172.21.15.153:3300/0 conn(0x7f969415fca0 0x7f96940d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 -- 172.21.15.27:0/1527520129 >> 172.21.15.27:0/1527520129 conn(0x7f969406c150 msgr2=0x7f969406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 -- 172.21.15.27:0/1527520129 shutdown_connections 2023-07-06T20:15:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.477+0000 7f969b1ff700 1 -- 172.21.15.27:0/1527520129 wait complete. 2023-07-06T20:15:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.479+0000 7f969b1ff700 1 Processor -- start 2023-07-06T20:15:17.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.479+0000 7f969b1ff700 1 -- start start 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969b1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b5c90 0x7f969414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969b1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96940b6110 0x7f96940c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969b1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 0x7f969415a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969b1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969414e2d0 con 0x7f969415fca0 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969b1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96940c72a0 con 0x7f96940b6110 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969b1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96940c7580 con 0x7f96940b5c90 2023-07-06T20:15:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969979c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 0x7f969415a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969979c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 0x7f969415a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45580/0 (socket says 172.21.15.27:45580) 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f969979c700 1 -- 172.21.15.27:0/2917078958 learned_addr learned my addr 172.21.15.27:0/2917078958 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f9693fff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96940b6110 0x7f96940c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.480+0000 7f9698f9b700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b5c90 0x7f969414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969979c700 1 -- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b5c90 msgr2=0x7f969414dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969979c700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b5c90 0x7f969414dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969979c700 1 -- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96940b6110 msgr2=0x7f96940c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969979c700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96940b6110 0x7f96940c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969979c700 1 -- 172.21.15.27:0/2917078958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c009a70 con 0x7f969415fca0 2023-07-06T20:15:17.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969979c700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 0x7f969415a530 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f96940cd8b0 tx=0x7f968c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:17.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c012430 con 0x7f969415fca0 2023-07-06T20:15:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c020070 con 0x7f969415fca0 2023-07-06T20:15:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c004280 con 0x7f969415fca0 2023-07-06T20:15:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.481+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96940c7860 con 0x7f969415fca0 2023-07-06T20:15:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.482+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96940b3ca0 con 0x7f969415fca0 2023-07-06T20:15:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.483+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96940008d0 con 0x7f969415fca0 2023-07-06T20:15:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.484+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f968c0043e0 con 0x7f969415fca0 2023-07-06T20:15:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.485+0000 7f96917fa700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f967806ace0 0x7f967806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.485+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f968c022070 con 0x7f969415fca0 2023-07-06T20:15:17.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.485+0000 7f9698f9b700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f967806ace0 0x7f967806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.486+0000 7f9698f9b700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f967806ace0 0x7f967806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9688009b40 tx=0x7f9688009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.489+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f968c056140 con 0x7f969415fca0 2023-07-06T20:15:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.696+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96940b6f90 con 0x7f969415fca0 2023-07-06T20:15:17.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.697+0000 7f96917fa700 1 -- 172.21.15.27:0/2917078958 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18340 (secure 0 0 0) 0x7f968c02c090 con 0x7f969415fca0 2023-07-06T20:15:17.699 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:17.700 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:16.713792+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:17.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f967806ace0 msgr2=0x7f967806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f967806ace0 0x7f967806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9688009b40 tx=0x7f9688009210 comp rx=0 tx=0).stop 2023-07-06T20:15:17.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 msgr2=0x7f969415a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 0x7f969415a530 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f96940cd8b0 tx=0x7f968c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 shutdown_connections 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f967806ace0 0x7f967806d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f969415fca0 0x7f969415a530 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f96940b5c90 0x7f969414dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 --2- 172.21.15.27:0/2917078958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96940b6110 0x7f96940c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 >> 172.21.15.27:0/2917078958 conn(0x7f969406c150 msgr2=0x7f96940b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:17.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.699+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 shutdown_connections 2023-07-06T20:15:17.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.700+0000 7f969b1ff700 1 -- 172.21.15.27:0/2917078958 wait complete. 2023-07-06T20:15:17.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.921+0000 7f289b5ff700 1 Processor -- start 2023-07-06T20:15:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.921+0000 7f289b5ff700 1 -- start start 2023-07-06T20:15:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.922+0000 7f289b5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.922+0000 7f289b5ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f28940b60e0 0x7f28940d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.922+0000 7f289b5ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940cf780 con 0x7f289415fca0 2023-07-06T20:15:17.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.922+0000 7f289b5ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28940cd490 con 0x7f28940b60e0 2023-07-06T20:15:17.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.922+0000 7f289b5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940cd770 con 0x7f28940b5c90 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2898b9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f28940b60e0 0x7f28940d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f28940b60e0 0x7f28940d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:38518/0 (socket says 172.21.15.27:38518) 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 -- 172.21.15.27:0/2129643396 learned_addr learned my addr 172.21.15.27:0/2129643396 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f289939b700 1 --1- 172.21.15.27:0/2129643396 >> v1:172.21.15.153:6789/0 conn(0x7f289415fca0 0x7f28940d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:37356/0 (socket says 172.21.15.27:37356) 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 -- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 msgr2=0x7f28940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 --2- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 -- 172.21.15.27:0/2129643396 >> v1:172.21.15.153:6789/0 conn(0x7f289415fca0 legacy=0x7f28940d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.923+0000 7f2899b9c700 1 -- 172.21.15.27:0/2129643396 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940cda50 con 0x7f28940b60e0 2023-07-06T20:15:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.924+0000 7f2899b9c700 1 --2- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940d3280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f288c009e50 tx=0x7f288c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70c1fd31dcdfb89 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.924+0000 7f288bfff700 1 -- 172.21.15.27:0/2129643396 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00e070 con 0x7f28940b60e0 2023-07-06T20:15:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.924+0000 7f288bfff700 1 -- 172.21.15.27:0/2129643396 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c00b910 con 0x7f28940b60e0 2023-07-06T20:15:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.924+0000 7f288bfff700 1 -- 172.21.15.27:0/2129643396 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00bcc0 con 0x7f28940b60e0 2023-07-06T20:15:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 -- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 msgr2=0x7f28940d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 --2- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940d3280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f288c009e50 tx=0x7f288c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 -- 172.21.15.27:0/2129643396 shutdown_connections 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 --2- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940d3280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 --2- 172.21.15.27:0/2129643396 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 -- 172.21.15.27:0/2129643396 >> 172.21.15.27:0/2129643396 conn(0x7f289406c150 msgr2=0x7f289406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 -- 172.21.15.27:0/2129643396 shutdown_connections 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.925+0000 7f289b5ff700 1 -- 172.21.15.27:0/2129643396 wait complete. 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.927+0000 7f289b5ff700 1 Processor -- start 2023-07-06T20:15:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.927+0000 7f289b5ff700 1 -- start start 2023-07-06T20:15:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289b5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289b5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289b5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 0x7f28940cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289b5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289415e6a0 con 0x7f289415fca0 2023-07-06T20:15:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289b5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d1440 con 0x7f28940b60e0 2023-07-06T20:15:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289b5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289415da90 con 0x7f28940b5c90 2023-07-06T20:15:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f2899b9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 0x7f28940cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f2899b9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 0x7f28940cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45596/0 (socket says 172.21.15.27:45596) 2023-07-06T20:15:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f2899b9c700 1 -- 172.21.15.27:0/1849998771 learned_addr learned my addr 172.21.15.27:0/1849998771 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f2898b9a700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.928+0000 7f289939b700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2899b9c700 1 -- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 msgr2=0x7f28940d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2899b9c700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2899b9c700 1 -- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 msgr2=0x7f28940c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2899b9c700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2899b9c700 1 -- 172.21.15.27:0/1849998771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c009b30 con 0x7f289415fca0 2023-07-06T20:15:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2899b9c700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 0x7f28940cbf20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f288c012000 tx=0x7f288c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c004800 con 0x7f289415fca0 2023-07-06T20:15:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c004960 con 0x7f289415fca0 2023-07-06T20:15:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c01c680 con 0x7f289415fca0 2023-07-06T20:15:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289415dd70 con 0x7f289415fca0 2023-07-06T20:15:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.929+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289415e330 con 0x7f289415fca0 2023-07-06T20:15:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.930+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28940008d0 con 0x7f289415fca0 2023-07-06T20:15:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.932+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f288c01c7e0 con 0x7f289415fca0 2023-07-06T20:15:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.932+0000 7f2889ffb700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f287806ace0 0x7f287806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.933+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f288c08ad60 con 0x7f289415fca0 2023-07-06T20:15:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.933+0000 7f289939b700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f287806ace0 0x7f287806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:17.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.934+0000 7f289939b700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f287806ace0 0x7f287806d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2890000f60 tx=0x7f2890007ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:17.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:17.937+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f288c055e00 con 0x7f289415fca0 2023-07-06T20:15:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.144+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28940b83f0 con 0x7f289415fca0 2023-07-06T20:15:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.146+0000 7f2889ffb700 1 -- 172.21.15.27:0/1849998771 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18340 (secure 0 0 0) 0x7f288c059420 con 0x7f289415fca0 2023-07-06T20:15:18.148 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:18.148 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:16.713792+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.148+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f287806ace0 msgr2=0x7f287806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.148+0000 7f289b5ff700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f287806ace0 0x7f287806d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2890000f60 tx=0x7f2890007ed0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.148+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 msgr2=0x7f28940cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.148+0000 7f289b5ff700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 0x7f28940cbf20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f288c012000 tx=0x7f288c004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.148+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 shutdown_connections 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.148+0000 7f289b5ff700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f287806ace0 0x7f287806d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.149+0000 7f289b5ff700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289415fca0 0x7f28940cbf20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.149+0000 7f289b5ff700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28940b5c90 0x7f28940d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.149+0000 7f289b5ff700 1 --2- 172.21.15.27:0/1849998771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28940b60e0 0x7f28940c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.149+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 >> 172.21.15.27:0/1849998771 conn(0x7f289406c150 msgr2=0x7f289415ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.149+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 shutdown_connections 2023-07-06T20:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.149+0000 7f289b5ff700 1 -- 172.21.15.27:0/1849998771 wait complete. 2023-07-06T20:15:18.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T20:15:18.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.378+0000 7fcd74dff700 1 Processor -- start 2023-07-06T20:15:18.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.378+0000 7fcd74dff700 1 -- start start 2023-07-06T20:15:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd74dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b5c90 0x7fcd700d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd74dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcd700b60e0 0x7fcd700d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd74dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700cf780 con 0x7fcd7015fca0 2023-07-06T20:15:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd74dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700cd490 con 0x7fcd700b60e0 2023-07-06T20:15:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd74dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700cd770 con 0x7fcd700b5c90 2023-07-06T20:15:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd6e59c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcd7015fca0 0x7fcd700d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40668/0 (socket says 172.21.15.27:40668) 2023-07-06T20:15:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd6e59c700 1 -- 172.21.15.27:0/1886395859 learned_addr learned my addr 172.21.15.27:0/1886395859 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.379+0000 7fcd6ed9d700 1 --2- 172.21.15.27:0/1886395859 >> v2:172.21.15.27:3300/0 conn(0x7fcd700b60e0 0x7fcd700d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd67fff700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b5c90 0x7fcd700d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6dd9b700 1 -- 172.21.15.27:0/1886395859 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180617703 0 0) 0x7fcd700cf780 con 0x7fcd7015fca0 2023-07-06T20:15:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6dd9b700 1 -- 172.21.15.27:0/1886395859 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd5c0035b0 con 0x7fcd7015fca0 2023-07-06T20:15:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6ed9d700 1 -- 172.21.15.27:0/1886395859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b5c90 msgr2=0x7fcd700d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6ed9d700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b5c90 0x7fcd700d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6ed9d700 1 -- 172.21.15.27:0/1886395859 >> v1:172.21.15.27:6789/0 conn(0x7fcd7015fca0 legacy=0x7fcd700d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6ed9d700 1 -- 172.21.15.27:0/1886395859 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700cda50 con 0x7fcd700b60e0 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd67fff700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b5c90 0x7fcd700d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6ed9d700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b60e0 0x7fcd700d3280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d6271ab3eb9925 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6dd9b700 1 -- 172.21.15.27:0/1886395859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800e070 con 0x7fcd700b60e0 2023-07-06T20:15:18.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6dd9b700 1 -- 172.21.15.27:0/1886395859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800bb20 con 0x7fcd700b60e0 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.380+0000 7fcd6dd9b700 1 -- 172.21.15.27:0/1886395859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68012430 con 0x7fcd700b60e0 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 -- 172.21.15.27:0/1886395859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b60e0 msgr2=0x7fcd700d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b60e0 0x7fcd700d3280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6800b330 comp rx=0 tx=0).stop 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 -- 172.21.15.27:0/1886395859 shutdown_connections 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b60e0 0x7fcd700d3280 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 --2- 172.21.15.27:0/1886395859 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b5c90 0x7fcd700d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 -- 172.21.15.27:0/1886395859 >> 172.21.15.27:0/1886395859 conn(0x7fcd7006c150 msgr2=0x7fcd7006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:18.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 -- 172.21.15.27:0/1886395859 shutdown_connections 2023-07-06T20:15:18.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.381+0000 7fcd74dff700 1 -- 172.21.15.27:0/1886395859 wait complete. 2023-07-06T20:15:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.383+0000 7fcd74dff700 1 Processor -- start 2023-07-06T20:15:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.383+0000 7fcd74dff700 1 -- start start 2023-07-06T20:15:18.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd74dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 0x7fcd70153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd74dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b60e0 0x7fcd700cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd74dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7015fca0 0x7fcd700cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd74dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70154070 con 0x7fcd700b5c90 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd74dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70153b40 con 0x7fcd7015fca0 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd74dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70153d20 con 0x7fcd700b60e0 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd6e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 0x7fcd70153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd6e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 0x7fcd70153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45618/0 (socket says 172.21.15.27:45618) 2023-07-06T20:15:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd6e59c700 1 -- 172.21.15.27:0/3206175133 learned_addr learned my addr 172.21.15.27:0/3206175133 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.384+0000 7fcd67fff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b60e0 0x7fcd700cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6ed9d700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7015fca0 0x7fcd700cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6e59c700 1 -- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b60e0 msgr2=0x7fcd700cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6e59c700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b60e0 0x7fcd700cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6e59c700 1 -- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7015fca0 msgr2=0x7fcd700cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6e59c700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7015fca0 0x7fcd700cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6e59c700 1 -- 172.21.15.27:0/3206175133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009a70 con 0x7fcd700b5c90 2023-07-06T20:15:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd6e59c700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 0x7fcd70153600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd60001eb0 tx=0x7fcd6000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60005d70 con 0x7fcd700b5c90 2023-07-06T20:15:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000ce50 con 0x7fcd700b5c90 2023-07-06T20:15:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600072f0 con 0x7fcd700b5c90 2023-07-06T20:15:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.385+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd700cc420 con 0x7fcd700b5c90 2023-07-06T20:15:18.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.386+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd700cc9e0 con 0x7fcd700b5c90 2023-07-06T20:15:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.386+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd700ccb70 con 0x7fcd700b5c90 2023-07-06T20:15:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.387+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd60007450 con 0x7fcd700b5c90 2023-07-06T20:15:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.388+0000 7fcd66ffd700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcd5006ace0 0x7fcd5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.389+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd60088830 con 0x7fcd700b5c90 2023-07-06T20:15:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.389+0000 7fcd67fff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcd5006ace0 0x7fcd5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.389+0000 7fcd67fff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcd5006ace0 0x7fcd5006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd7015ef50 tx=0x7fcd58008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.393+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcd600538d0 con 0x7fcd700b5c90 2023-07-06T20:15:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:18.647+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcd700008d0 con 0x7fcd700b5c90 2023-07-06T20:15:19.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.238+0000 7fcd66ffd700 1 -- 172.21.15.27:0/3206175133 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcd60056ef0 con 0x7fcd700b5c90 2023-07-06T20:15:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcd5006ace0 msgr2=0x7fcd5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcd5006ace0 0x7fcd5006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd7015ef50 tx=0x7fcd58008040 comp rx=0 tx=0).stop 2023-07-06T20:15:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 msgr2=0x7fcd70153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 0x7fcd70153600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd60001eb0 tx=0x7fcd6000c7b0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 shutdown_connections 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcd5006ace0 0x7fcd5006d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd7015fca0 0x7fcd700cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd700b5c90 0x7fcd70153600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 --2- 172.21.15.27:0/3206175133 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcd700b60e0 0x7fcd700cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 >> 172.21.15.27:0/3206175133 conn(0x7fcd7006c150 msgr2=0x7fcd70159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 shutdown_connections 2023-07-06T20:15:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.241+0000 7fcd74dff700 1 -- 172.21.15.27:0/3206175133 wait complete. 2023-07-06T20:15:19.262 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T20:15:19.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T20:15:19.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.470+0000 7efee83ff700 1 Processor -- start 2023-07-06T20:15:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.470+0000 7efee83ff700 1 -- start start 2023-07-06T20:15:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee83ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efee015fca0 0x7efee00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee83ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 0x7efee00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee83ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee00cf780 con 0x7efee00b60e0 2023-07-06T20:15:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee83ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00cd490 con 0x7efee015fca0 2023-07-06T20:15:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee83ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00cd770 con 0x7efee00b5c90 2023-07-06T20:15:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee699c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efee00b60e0 0x7efee00cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40676/0 (socket says 172.21.15.27:40676) 2023-07-06T20:15:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.471+0000 7efee699c700 1 -- 172.21.15.27:0/2840811002 learned_addr learned my addr 172.21.15.27:0/2840811002 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee599a700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 0x7efee00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee619b700 1 --2- 172.21.15.27:0/2840811002 >> v2:172.21.15.153:3300/0 conn(0x7efee015fca0 0x7efee00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200769929 0 0) 0x7efee00cf780 con 0x7efee00b60e0 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efecc0035b0 con 0x7efee00b60e0 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2338312863 0 0) 0x7efecc0035b0 con 0x7efee00b60e0 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 msgr2=0x7efee00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 0x7efee00d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 >> v2:172.21.15.153:3300/0 conn(0x7efee015fca0 msgr2=0x7efee00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 --2- 172.21.15.27:0/2840811002 >> v2:172.21.15.153:3300/0 conn(0x7efee015fca0 0x7efee00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00cda50 con 0x7efee00b60e0 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7efed80019f0 con 0x7efee00b60e0 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 >> v1:172.21.15.27:6789/0 conn(0x7efee00b60e0 legacy=0x7efee00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 0x7efecc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc003f80 0x7efecc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc004d90 0x7efecc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00cd490 con 0x7efecc003f80 2023-07-06T20:15:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00cd770 con 0x7efecc004d90 2023-07-06T20:15:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.472+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc00ba10 con 0x7efee00b5c90 2023-07-06T20:15:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc003f80 0x7efecc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee619b700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 0x7efecc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee699c700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc004d90 0x7efecc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 -- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 msgr2=0x7efecc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:19.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efee00b5c90 0x7efecc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 -- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc004d90 msgr2=0x7efecc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc004d90 0x7efecc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 -- 172.21.15.27:0/2840811002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0035b0 con 0x7efecc003f80 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee699c700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efecc004d90 0x7efecc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee599a700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc003f80 0x7efecc004850 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efed0009b60 tx=0x7efecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d2a23579e4fb65 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000e6c0 con 0x7efecc003f80 2023-07-06T20:15:19.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.473+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000e820 con 0x7efecc003f80 2023-07-06T20:15:19.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee5199700 1 -- 172.21.15.27:0/2840811002 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000eb90 con 0x7efecc003f80 2023-07-06T20:15:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee83ff700 1 -- 172.21.15.27:0/2840811002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc003f80 msgr2=0x7efecc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee83ff700 1 --2- 172.21.15.27:0/2840811002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efecc003f80 0x7efecc004850 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efed0009b60 tx=0x7efecc005880 comp rx=0 tx=0).stop 2023-07-06T20:15:19.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee699c700 1 -- 172.21.15.27:0/2840811002 reap_dead start 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee83ff700 1 -- 172.21.15.27:0/2840811002 shutdown_connections 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee83ff700 1 -- 172.21.15.27:0/2840811002 >> 172.21.15.27:0/2840811002 conn(0x7efee006c150 msgr2=0x7efee006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee83ff700 1 -- 172.21.15.27:0/2840811002 shutdown_connections 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.474+0000 7efee83ff700 1 -- 172.21.15.27:0/2840811002 wait complete. 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.476+0000 7efee83ff700 1 Processor -- start 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.476+0000 7efee83ff700 1 -- start start 2023-07-06T20:15:19.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.477+0000 7efee83ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efecc003f80 0x7efee0150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.477+0000 7efee83ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0153230 0x7efee00cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.477+0000 7efee83ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 0x7efee0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.477+0000 7efee83ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01536a0 con 0x7efee0155060 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.477+0000 7efee83ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00ca0b0 con 0x7efee0153230 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.477+0000 7efee83ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00ca390 con 0x7efecc003f80 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 0x7efee0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 0x7efee0158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45642/0 (socket says 172.21.15.27:45642) 2023-07-06T20:15:19.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 -- 172.21.15.27:0/369727812 learned_addr learned my addr 172.21.15.27:0/369727812 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee619b700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efecc003f80 0x7efee0150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee599a700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0153230 0x7efee00cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 -- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efecc003f80 msgr2=0x7efee0150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efecc003f80 0x7efee0150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 -- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0153230 msgr2=0x7efee00cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0153230 0x7efee00cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:19.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 -- 172.21.15.27:0/369727812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00ca670 con 0x7efee0155060 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee619b700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efecc003f80 0x7efee0150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee599a700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0153230 0x7efee00cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efee699c700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 0x7efee0158ec0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efee00ce560 tx=0x7efed800e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.478+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800ed80 con 0x7efee0155060 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.479+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed8013420 con 0x7efee0155060 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.479+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee00ca950 con 0x7efee0155060 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.479+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800eee0 con 0x7efee0155060 2023-07-06T20:15:19.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.479+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01554e0 con 0x7efee0155060 2023-07-06T20:15:19.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.480+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee00008d0 con 0x7efee0155060 2023-07-06T20:15:19.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.481+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efed80137b0 con 0x7efee0155060 2023-07-06T20:15:19.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.482+0000 7efed6ffd700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efec406ace0 0x7efec406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:19.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.482+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efed8089b70 con 0x7efee0155060 2023-07-06T20:15:19.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.482+0000 7efee619b700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efec406ace0 0x7efec406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:19.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.483+0000 7efee619b700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efec406ace0 0x7efec406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efedc009bd0 tx=0x7efedc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:19.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.486+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efed8054c20 con 0x7efee0155060 2023-07-06T20:15:19.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:19.738+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efee00cc530 con 0x7efee0155060 2023-07-06T20:15:20.098 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-07-06T20:15:20.096+0000 7f3c53ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T20:15:20.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.240+0000 7efed6ffd700 1 -- 172.21.15.27:0/369727812 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7efed8058240 con 0x7efee0155060 2023-07-06T20:15:20.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.242+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efec406ace0 msgr2=0x7efec406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.242+0000 7efee83ff700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efec406ace0 0x7efec406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efedc009bd0 tx=0x7efedc008040 comp rx=0 tx=0).stop 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.242+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 msgr2=0x7efee0158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.242+0000 7efee83ff700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 0x7efee0158ec0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efee00ce560 tx=0x7efed800e4f0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 shutdown_connections 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7efec406ace0 0x7efec406d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efee0155060 0x7efee0158ec0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efecc003f80 0x7efee0150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 --2- 172.21.15.27:0/369727812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0153230 0x7efee00cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 >> 172.21.15.27:0/369727812 conn(0x7efee006c150 msgr2=0x7efee0160070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:20.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 shutdown_connections 2023-07-06T20:15:20.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.243+0000 7efee83ff700 1 -- 172.21.15.27:0/369727812 wait complete. 2023-07-06T20:15:20.264 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T20:15:20.265 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:20.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.467+0000 7faf2a9ff700 1 Processor -- start 2023-07-06T20:15:20.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.468+0000 7faf2a9ff700 1 -- start start 2023-07-06T20:15:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf2a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 0x7faf240cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf2a9ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf240cf780 con 0x7faf2415fca0 2023-07-06T20:15:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf2a9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf240cd490 con 0x7faf240b5c90 2023-07-06T20:15:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf2a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cd770 con 0x7faf240b6110 2023-07-06T20:15:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf23fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf2415fca0 0x7faf240d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40686/0 (socket says 172.21.15.27:40686) 2023-07-06T20:15:20.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf23fff700 1 -- 172.21.15.27:0/2052704463 learned_addr learned my addr 172.21.15.27:0/2052704463 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.469+0000 7faf28f9c700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 0x7faf240cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275078074 0 0) 0x7faf240cf780 con 0x7faf2415fca0 2023-07-06T20:15:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c0035b0 con 0x7faf2415fca0 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810314988 0 0) 0x7faf240cd490 con 0x7faf240b5c90 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf240cf780 con 0x7faf240b5c90 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 5017401 0 0) 0x7faf0c0035b0 con 0x7faf2415fca0 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 msgr2=0x7faf240cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 0x7faf240cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 >> v1:172.21.15.153:6789/0 conn(0x7faf240b5c90 legacy=0x7faf240d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240cda50 con 0x7faf2415fca0 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7faf14001670 con 0x7faf2415fca0 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 >> v1:172.21.15.27:6789/0 conn(0x7faf2415fca0 legacy=0x7faf240d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf0c003bc0 0x7faf0c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 0x7faf0c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf0c004d90 0x7faf0c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cd770 con 0x7faf0c004d90 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c00bac0 con 0x7faf0c003bc0 2023-07-06T20:15:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.470+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c00bc30 con 0x7faf240b6110 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf0c004d90 0x7faf0c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf23fff700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf0c003bc0 0x7faf0c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf237fe700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 0x7faf0c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 -- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 msgr2=0x7faf0c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf240b6110 0x7faf0c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 -- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf0c003bc0 msgr2=0x7faf0c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf0c003bc0 0x7faf0c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 -- 172.21.15.27:0/2052704463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0035b0 con 0x7faf0c004d90 2023-07-06T20:15:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.471+0000 7faf28f9c700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf0c004d90 0x7faf0c005200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009fc0 tx=0x7faf0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbedd5ddc733de41 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00ebd0 con 0x7faf0c004d90 2023-07-06T20:15:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c00f040 con 0x7faf0c004d90 2023-07-06T20:15:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf22ffd700 1 -- 172.21.15.27:0/2052704463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c014480 con 0x7faf0c004d90 2023-07-06T20:15:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf2a9ff700 1 -- 172.21.15.27:0/2052704463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf0c004d90 msgr2=0x7faf0c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/2052704463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf0c004d90 0x7faf0c005200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009fc0 tx=0x7faf0c005880 comp rx=0 tx=0).stop 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf28f9c700 1 -- 172.21.15.27:0/2052704463 reap_dead start 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf2a9ff700 1 -- 172.21.15.27:0/2052704463 shutdown_connections 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf2a9ff700 1 -- 172.21.15.27:0/2052704463 >> 172.21.15.27:0/2052704463 conn(0x7faf2406c150 msgr2=0x7faf2406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf2a9ff700 1 -- 172.21.15.27:0/2052704463 shutdown_connections 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.472+0000 7faf2a9ff700 1 -- 172.21.15.27:0/2052704463 wait complete. 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.475+0000 7faf2a9ff700 1 Processor -- start 2023-07-06T20:15:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.475+0000 7faf2a9ff700 1 -- start start 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf2a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf2414d130 0x7faf240c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf2a9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 0x7faf2415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf2a9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf241578f0 0x7faf2415d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf2a9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240ca810 con 0x7faf240cb8b0 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf2a9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240caaf0 con 0x7faf241578f0 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf2a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cadd0 con 0x7faf2414d130 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.476+0000 7faf237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 0x7faf2415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 0x7faf2415d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45670/0 (socket says 172.21.15.27:45670) 2023-07-06T20:15:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 -- 172.21.15.27:0/3345399754 learned_addr learned my addr 172.21.15.27:0/3345399754 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf28f9c700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf241578f0 0x7faf2415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf23fff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf2414d130 0x7faf240c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 -- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf2414d130 msgr2=0x7faf240c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf2414d130 0x7faf240c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 -- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf241578f0 msgr2=0x7faf2415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf241578f0 0x7faf2415d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 -- 172.21.15.27:0/3345399754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240cb0b0 con 0x7faf240cb8b0 2023-07-06T20:15:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf23fff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf2414d130 0x7faf240c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.477+0000 7faf237fe700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 0x7faf2415d0d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf1800f7f0 tx=0x7faf1800fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.478+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180052e0 con 0x7faf240cb8b0 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.478+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf18005440 con 0x7faf240cb8b0 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.478+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180038c0 con 0x7faf240cb8b0 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.478+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf240cb390 con 0x7faf240cb8b0 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.478+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2414d500 con 0x7faf240cb8b0 2023-07-06T20:15:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.479+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf240008d0 con 0x7faf240cb8b0 2023-07-06T20:15:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.480+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf18003a20 con 0x7faf240cb8b0 2023-07-06T20:15:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.481+0000 7faf20ff9700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7faf1006ace0 0x7faf1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.481+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf18089370 con 0x7faf240cb8b0 2023-07-06T20:15:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.481+0000 7faf23fff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7faf1006ace0 0x7faf1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.482+0000 7faf23fff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7faf1006ace0 0x7faf1006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf140048d0 tx=0x7faf14009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:20.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.485+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf18054420 con 0x7faf240cb8b0 2023-07-06T20:15:20.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:15:20.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:15:20.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.736+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf2415a4f0 con 0x7faf240cb8b0 2023-07-06T20:15:20.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.736+0000 7faf20ff9700 1 -- 172.21.15.27:0/3345399754 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4671 (secure 0 0 0) 0x7faf18057a40 con 0x7faf240cb8b0 2023-07-06T20:15:20.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:20.738 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:15:16.714161+0000","modified":"2023-07-06T20:15:20.237603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/827990525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":827990525},{"type":"v1","addr":"172.21.15.27:6835","nonce":827990525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T20:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7faf1006ace0 msgr2=0x7faf1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7faf1006ace0 0x7faf1006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf140048d0 tx=0x7faf14009040 comp rx=0 tx=0).stop 2023-07-06T20:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 msgr2=0x7faf2415d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 0x7faf2415d0d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faf1800f7f0 tx=0x7faf1800fb00 comp rx=0 tx=0).stop 2023-07-06T20:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 shutdown_connections 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7faf1006ace0 0x7faf1006d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf241578f0 0x7faf2415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf2414d130 0x7faf240c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 --2- 172.21.15.27:0/3345399754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf240cb8b0 0x7faf2415d0d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 >> 172.21.15.27:0/3345399754 conn(0x7faf2406c150 msgr2=0x7faf240b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 shutdown_connections 2023-07-06T20:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.739+0000 7faf2a9ff700 1 -- 172.21.15.27:0/3345399754 wait complete. 2023-07-06T20:15:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2023-07-06T20:15:20.760 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.962+0000 7f854bdff700 1 Processor -- start 2023-07-06T20:15:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.962+0000 7f854bdff700 1 -- start start 2023-07-06T20:15:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854bdff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f854406c330 0x7f854406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854bdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85440c9010 0x7f85440c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854bdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85441565f0 0x7f8544156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854bdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854406cc40 con 0x7f85441565f0 2023-07-06T20:15:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854bdff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85440c9a60 con 0x7f854406c330 2023-07-06T20:15:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854bdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8544159b70 con 0x7f85440c9010 2023-07-06T20:15:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854a39c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85441565f0 0x7f8544156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854a39c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85441565f0 0x7f8544156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45672/0 (socket says 172.21.15.27:45672) 2023-07-06T20:15:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.963+0000 7f854a39c700 1 -- 172.21.15.27:0/4140379964 learned_addr learned my addr 172.21.15.27:0/4140379964 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:20.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f854a39c700 1 -- 172.21.15.27:0/4140379964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85440c9010 msgr2=0x7f85440c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:20.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f854a39c700 1 --2- 172.21.15.27:0/4140379964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85440c9010 0x7f85440c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f854a39c700 1 -- 172.21.15.27:0/4140379964 >> v2:172.21.15.153:3300/0 conn(0x7f854406c330 msgr2=0x7f854406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f854a39c700 1 --2- 172.21.15.27:0/4140379964 >> v2:172.21.15.153:3300/0 conn(0x7f854406c330 0x7f854406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f854a39c700 1 -- 172.21.15.27:0/4140379964 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544159e50 con 0x7f85441565f0 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f854a39c700 1 --2- 172.21.15.27:0/4140379964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85441565f0 0x7f8544156a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f854406f160 tx=0x7f853c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae62199c9b8443b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f8548b99700 1 -- 172.21.15.27:0/4140379964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c00e070 con 0x7f85441565f0 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f8548b99700 1 -- 172.21.15.27:0/4140379964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c00b9c0 con 0x7f85441565f0 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.964+0000 7f8548b99700 1 -- 172.21.15.27:0/4140379964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c00bd70 con 0x7f85441565f0 2023-07-06T20:15:20.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 -- 172.21.15.27:0/4140379964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85441565f0 msgr2=0x7f8544156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 --2- 172.21.15.27:0/4140379964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85441565f0 0x7f8544156a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f854406f160 tx=0x7f853c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 -- 172.21.15.27:0/4140379964 shutdown_connections 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 --2- 172.21.15.27:0/4140379964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85441565f0 0x7f8544156a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 --2- 172.21.15.27:0/4140379964 >> v2:172.21.15.153:3300/0 conn(0x7f854406c330 0x7f854406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 --2- 172.21.15.27:0/4140379964 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85440c9010 0x7f85440c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 -- 172.21.15.27:0/4140379964 >> 172.21.15.27:0/4140379964 conn(0x7f85440cb440 msgr2=0x7f85440cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 -- 172.21.15.27:0/4140379964 shutdown_connections 2023-07-06T20:15:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.965+0000 7f854bdff700 1 -- 172.21.15.27:0/4140379964 wait complete. 2023-07-06T20:15:20.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.967+0000 7f854bdff700 1 Processor -- start 2023-07-06T20:15:20.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.968+0000 7f854bdff700 1 -- start start 2023-07-06T20:15:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854bdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 0x7f85440b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854bdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85440c9010 0x7f8544158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854bdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85441565f0 0x7f85440b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854bdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440c5f70 con 0x7f854406c330 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854bdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440c6250 con 0x7f85440c9010 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854bdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85440c6530 con 0x7f85441565f0 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f8549b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 0x7f85440b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f8549b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 0x7f85440b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45686/0 (socket says 172.21.15.27:45686) 2023-07-06T20:15:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f8549b9b700 1 -- 172.21.15.27:0/947607456 learned_addr learned my addr 172.21.15.27:0/947607456 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854a39c700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85441565f0 0x7f85440b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.969+0000 7f854939a700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85440c9010 0x7f8544158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f8549b9b700 1 -- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85441565f0 msgr2=0x7f85440b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f8549b9b700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85441565f0 0x7f85440b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f8549b9b700 1 -- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85440c9010 msgr2=0x7f8544158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f8549b9b700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85440c9010 0x7f8544158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:20.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f8549b9b700 1 -- 172.21.15.27:0/947607456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c009b30 con 0x7f854406c330 2023-07-06T20:15:20.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f8549b9b700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 0x7f85440b42a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f854000b4a0 tx=0x7f854000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:20.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540011830 con 0x7f854406c330 2023-07-06T20:15:20.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540011990 con 0x7f854406c330 2023-07-06T20:15:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540011d40 con 0x7f854406c330 2023-07-06T20:15:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85440c6810 con 0x7f854406c330 2023-07-06T20:15:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.970+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85440c75e0 con 0x7f854406c330 2023-07-06T20:15:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.971+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85440008d0 con 0x7f854406c330 2023-07-06T20:15:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.972+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f854001a400 con 0x7f854406c330 2023-07-06T20:15:20.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.973+0000 7f853a7fc700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f852806ace0 0x7f852806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:20.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.973+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8540089500 con 0x7f854406c330 2023-07-06T20:15:20.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.974+0000 7f854939a700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f852806ace0 0x7f852806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:20.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.975+0000 7f854939a700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f852806ace0 0x7f852806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8534006fd0 tx=0x7f8534008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:20.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:20.978+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8540054530 con 0x7f854406c330 2023-07-06T20:15:21.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.186+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85440cc2d0 con 0x7f854406c330 2023-07-06T20:15:21.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.187+0000 7f853a7fc700 1 -- 172.21.15.27:0/947607456 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f8540057b50 con 0x7f854406c330 2023-07-06T20:15:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:21.190 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:18.212485+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:21.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f852806ace0 msgr2=0x7f852806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f852806ace0 0x7f852806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8534006fd0 tx=0x7f8534008040 comp rx=0 tx=0).stop 2023-07-06T20:15:21.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 msgr2=0x7f85440b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 0x7f85440b42a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f854000b4a0 tx=0x7f854000b860 comp rx=0 tx=0).stop 2023-07-06T20:15:21.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 shutdown_connections 2023-07-06T20:15:21.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f852806ace0 0x7f852806d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85441565f0 0x7f85440b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f854406c330 0x7f85440b42a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 --2- 172.21.15.27:0/947607456 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85440c9010 0x7f8544158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.189+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 >> 172.21.15.27:0/947607456 conn(0x7f85440cb440 msgr2=0x7f854406fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:21.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.190+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 shutdown_connections 2023-07-06T20:15:21.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.190+0000 7f854bdff700 1 -- 172.21.15.27:0/947607456 wait complete. 2023-07-06T20:15:21.210 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T20:15:21.210 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:15:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.412+0000 7f3f61bff700 1 Processor -- start 2023-07-06T20:15:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.412+0000 7f3f61bff700 1 -- start start 2023-07-06T20:15:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f61bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f5c15dc80 0x7f3f5c160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f61bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c0d1250 0x7f3f5c160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f61bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c06cb60 con 0x7f3f5c15d8b0 2023-07-06T20:15:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f61bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c1613a0 con 0x7f3f5c15dc80 2023-07-06T20:15:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f61bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c06c230 con 0x7f3f5c0d1250 2023-07-06T20:15:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f5b7fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3f5c15d8b0 0x7f3f5c0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40696/0 (socket says 172.21.15.27:40696) 2023-07-06T20:15:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f5b7fe700 1 -- 172.21.15.27:0/3377451373 learned_addr learned my addr 172.21.15.27:0/3377451373 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f5bfff700 1 --2- 172.21.15.27:0/3377451373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c0d1250 0x7f3f5c160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f5affd700 1 --2- 172.21.15.27:0/3377451373 >> v2:172.21.15.153:3300/0 conn(0x7f3f5c15dc80 0x7f3f5c160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.414+0000 7f3f5a7fc700 1 -- 172.21.15.27:0/3377451373 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878535168 0 0) 0x7f3f5c06cb60 con 0x7f3f5c15d8b0 2023-07-06T20:15:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5a7fc700 1 -- 172.21.15.27:0/3377451373 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f440035b0 con 0x7f3f5c15d8b0 2023-07-06T20:15:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5bfff700 1 -- 172.21.15.27:0/3377451373 >> v2:172.21.15.153:3300/0 conn(0x7f3f5c15dc80 msgr2=0x7f3f5c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5bfff700 1 --2- 172.21.15.27:0/3377451373 >> v2:172.21.15.153:3300/0 conn(0x7f3f5c15dc80 0x7f3f5c160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5bfff700 1 -- 172.21.15.27:0/3377451373 >> v1:172.21.15.27:6789/0 conn(0x7f3f5c15d8b0 legacy=0x7f3f5c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5bfff700 1 -- 172.21.15.27:0/3377451373 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c06c510 con 0x7f3f5c0d1250 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5bfff700 1 --2- 172.21.15.27:0/3377451373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c0d1250 0x7f3f5c160b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009dc0 tx=0x7f3f5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912359a70b1abfb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5a7fc700 1 -- 172.21.15.27:0/3377451373 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400e070 con 0x7f3f5c0d1250 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5a7fc700 1 -- 172.21.15.27:0/3377451373 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400bae0 con 0x7f3f5c0d1250 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.415+0000 7f3f5a7fc700 1 -- 172.21.15.27:0/3377451373 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400be90 con 0x7f3f5c0d1250 2023-07-06T20:15:21.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 -- 172.21.15.27:0/3377451373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c0d1250 msgr2=0x7f3f5c160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 --2- 172.21.15.27:0/3377451373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c0d1250 0x7f3f5c160b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009dc0 tx=0x7f3f5400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 -- 172.21.15.27:0/3377451373 shutdown_connections 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 --2- 172.21.15.27:0/3377451373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c0d1250 0x7f3f5c160b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 --2- 172.21.15.27:0/3377451373 >> v2:172.21.15.153:3300/0 conn(0x7f3f5c15dc80 0x7f3f5c160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 -- 172.21.15.27:0/3377451373 >> 172.21.15.27:0/3377451373 conn(0x7f3f5c06db20 msgr2=0x7f3f5c06df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 -- 172.21.15.27:0/3377451373 shutdown_connections 2023-07-06T20:15:21.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.416+0000 7f3f61bff700 1 -- 172.21.15.27:0/3377451373 wait complete. 2023-07-06T20:15:21.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.418+0000 7f3f61bff700 1 Processor -- start 2023-07-06T20:15:21.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.418+0000 7f3f61bff700 1 -- start start 2023-07-06T20:15:21.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f61bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 0x7f3f5c0c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f61bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f5c15d8b0 0x7f3f5c1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f61bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c1f27d0 0x7f3f5c1f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f61bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c4a90 con 0x7f3f5c0d1250 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f61bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0ad970 con 0x7f3f5c15d8b0 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f61bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0b2620 con 0x7f3f5c1f27d0 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 0x7f3f5c0c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f5b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 0x7f3f5c0c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45702/0 (socket says 172.21.15.27:45702) 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f5b7fe700 1 -- 172.21.15.27:0/2208486313 learned_addr learned my addr 172.21.15.27:0/2208486313 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:21.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f5bfff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c1f27d0 0x7f3f5c1f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.419+0000 7f3f5affd700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f5c15d8b0 0x7f3f5c1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f5b7fe700 1 -- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c1f27d0 msgr2=0x7f3f5c1f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f5b7fe700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c1f27d0 0x7f3f5c1f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f5b7fe700 1 -- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f5c15d8b0 msgr2=0x7f3f5c1f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f5b7fe700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f5c15d8b0 0x7f3f5c1f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f5b7fe700 1 -- 172.21.15.27:0/2208486313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54009a70 con 0x7f3f5c0d1250 2023-07-06T20:15:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f5b7fe700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 0x7f3f5c0c7020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c730 tx=0x7f3f4c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:21.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c005d70 con 0x7f3f5c0d1250 2023-07-06T20:15:21.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c0070d0 con 0x7f3f5c0d1250 2023-07-06T20:15:21.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0b2900 con 0x7f3f5c0d1250 2023-07-06T20:15:21.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c007440 con 0x7f3f5c0d1250 2023-07-06T20:15:21.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.420+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c0b2ec0 con 0x7f3f5c0d1250 2023-07-06T20:15:21.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.421+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0adb40 con 0x7f3f5c0d1250 2023-07-06T20:15:21.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.422+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f4c0075a0 con 0x7f3f5c0d1250 2023-07-06T20:15:21.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.423+0000 7f3f43fff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f4806ace0 0x7f3f4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.423+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f4c088820 con 0x7f3f5c0d1250 2023-07-06T20:15:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.424+0000 7f3f5affd700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f4806ace0 0x7f3f4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.424+0000 7f3f5affd700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f4806ace0 0x7f3f4806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009b40 tx=0x7f3f50009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:21.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.428+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f4c0538e0 con 0x7f3f5c0d1250 2023-07-06T20:15:21.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.628+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f5c1610c0 con 0x7f3f4806ace0 2023-07-06T20:15:21.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.640+0000 7f3f43fff700 1 -- 172.21.15.27:0/2208486313 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311081 (secure 0 0 0) 0x7f3f50011810 con 0x7f3f4806ace0 2023-07-06T20:15:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:21.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f4806ace0 msgr2=0x7f3f4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f4806ace0 0x7f3f4806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009b40 tx=0x7f3f50009210 comp rx=0 tx=0).stop 2023-07-06T20:15:21.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 msgr2=0x7f3f5c0c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 0x7f3f5c0c7020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00c730 tx=0x7f3f4c00ca40 comp rx=0 tx=0).stop 2023-07-06T20:15:21.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 shutdown_connections 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f3f4806ace0 0x7f3f4806d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3f5c1f27d0 0x7f3f5c1f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f5c0d1250 0x7f3f5c0c7020 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 --2- 172.21.15.27:0/2208486313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f5c15d8b0 0x7f3f5c1f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 >> 172.21.15.27:0/2208486313 conn(0x7f3f5c06db20 msgr2=0x7f3f5c15fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 shutdown_connections 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.644+0000 7f3f61bff700 1 -- 172.21.15.27:0/2208486313 wait complete. 2023-07-06T20:15:21.648 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:15:21.664 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-06T20:15:20.362794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220012,"kb_used_data":4396,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754708,"statfs":{"total":773094113280,"available":772868820992,"internally_reserved":0,"allocated":4501504,"data_stored":2528895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.169091"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152170+0000","last_change":"2023-07-06T20:15:17.235418+0000","last_active":"2023-07-06T20:15:19.152170+0000","last_peered":"2023-07-06T20:15:19.152170+0000","last_clean":"2023-07-06T20:15:19.152170+0000","last_became_active":"2023-07-06T20:15:17.235083+0000","last_became_peered":"2023-07-06T20:15:17.235083+0000","last_unstale":"2023-07-06T20:15:19.152170+0000","last_undegraded":"2023-07-06T20:15:19.152170+0000","last_fullsized":"2023-07-06T20:15:19.152170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:35.832975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.151987+0000","last_change":"2023-07-06T20:15:17.229383+0000","last_active":"2023-07-06T20:15:19.151987+0000","last_peered":"2023-07-06T20:15:19.151987+0000","last_clean":"2023-07-06T20:15:19.151987+0000","last_became_active":"2023-07-06T20:15:17.229052+0000","last_became_peered":"2023-07-06T20:15:17.229052+0000","last_unstale":"2023-07-06T20:15:19.151987+0000","last_undegraded":"2023-07-06T20:15:19.151987+0000","last_fullsized":"2023-07-06T20:15:19.151987+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:27.294552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152730+0000","last_change":"2023-07-06T20:15:17.246105+0000","last_active":"2023-07-06T20:15:19.152730+0000","last_peered":"2023-07-06T20:15:19.152730+0000","last_clean":"2023-07-06T20:15:19.152730+0000","last_became_active":"2023-07-06T20:15:17.245834+0000","last_became_peered":"2023-07-06T20:15:17.245834+0000","last_unstale":"2023-07-06T20:15:19.152730+0000","last_undegraded":"2023-07-06T20:15:19.152730+0000","last_fullsized":"2023-07-06T20:15:19.152730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:57:48.651414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216966+0000","last_change":"2023-07-06T20:15:17.228851+0000","last_active":"2023-07-06T20:15:18.216966+0000","last_peered":"2023-07-06T20:15:18.216966+0000","last_clean":"2023-07-06T20:15:18.216966+0000","last_became_active":"2023-07-06T20:15:17.228586+0000","last_became_peered":"2023-07-06T20:15:17.228586+0000","last_unstale":"2023-07-06T20:15:18.216966+0000","last_undegraded":"2023-07-06T20:15:18.216966+0000","last_fullsized":"2023-07-06T20:15:18.216966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:49:20.404609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216657+0000","last_change":"2023-07-06T20:15:17.228308+0000","last_active":"2023-07-06T20:15:18.216657+0000","last_peered":"2023-07-06T20:15:18.216657+0000","last_clean":"2023-07-06T20:15:18.216657+0000","last_became_active":"2023-07-06T20:15:17.228012+0000","last_became_peered":"2023-07-06T20:15:17.228012+0000","last_unstale":"2023-07-06T20:15:18.216657+0000","last_undegraded":"2023-07-06T20:15:18.216657+0000","last_fullsized":"2023-07-06T20:15:18.216657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:04:45.731057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700882+0000","last_change":"2023-07-06T20:15:17.223324+0000","last_active":"2023-07-06T20:15:18.700882+0000","last_peered":"2023-07-06T20:15:18.700882+0000","last_clean":"2023-07-06T20:15:18.700882+0000","last_became_active":"2023-07-06T20:15:17.223065+0000","last_became_peered":"2023-07-06T20:15:17.223065+0000","last_unstale":"2023-07-06T20:15:18.700882+0000","last_undegraded":"2023-07-06T20:15:18.700882+0000","last_fullsized":"2023-07-06T20:15:18.700882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:51:28.509267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345389+0000","last_change":"2023-07-06T20:15:17.229972+0000","last_active":"2023-07-06T20:15:18.345389+0000","last_peered":"2023-07-06T20:15:18.345389+0000","last_clean":"2023-07-06T20:15:18.345389+0000","last_became_active":"2023-07-06T20:15:17.229670+0000","last_became_peered":"2023-07-06T20:15:17.229670+0000","last_unstale":"2023-07-06T20:15:18.345389+0000","last_undegraded":"2023-07-06T20:15:18.345389+0000","last_fullsized":"2023-07-06T20:15:18.345389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.909334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152386+0000","last_change":"2023-07-06T20:15:17.226201+0000","last_active":"2023-07-06T20:15:19.152386+0000","last_peered":"2023-07-06T20:15:19.152386+0000","last_clean":"2023-07-06T20:15:19.152386+0000","last_became_active":"2023-07-06T20:15:17.225906+0000","last_became_peered":"2023-07-06T20:15:17.225906+0000","last_unstale":"2023-07-06T20:15:19.152386+0000","last_undegraded":"2023-07-06T20:15:19.152386+0000","last_fullsized":"2023-07-06T20:15:19.152386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:59.691407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152873+0000","last_change":"2023-07-06T20:15:17.246977+0000","last_active":"2023-07-06T20:15:19.152873+0000","last_peered":"2023-07-06T20:15:19.152873+0000","last_clean":"2023-07-06T20:15:19.152873+0000","last_became_active":"2023-07-06T20:15:17.246690+0000","last_became_peered":"2023-07-06T20:15:17.246690+0000","last_unstale":"2023-07-06T20:15:19.152873+0000","last_undegraded":"2023-07-06T20:15:19.152873+0000","last_fullsized":"2023-07-06T20:15:19.152873+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:40:08.053320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.346007+0000","last_change":"2023-07-06T20:15:17.237522+0000","last_active":"2023-07-06T20:15:18.346007+0000","last_peered":"2023-07-06T20:15:18.346007+0000","last_clean":"2023-07-06T20:15:18.346007+0000","last_became_active":"2023-07-06T20:15:17.237289+0000","last_became_peered":"2023-07-06T20:15:17.237289+0000","last_unstale":"2023-07-06T20:15:18.346007+0000","last_undegraded":"2023-07-06T20:15:18.346007+0000","last_fullsized":"2023-07-06T20:15:18.346007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:30:07.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345735+0000","last_change":"2023-07-06T20:15:17.232247+0000","last_active":"2023-07-06T20:15:18.345735+0000","last_peered":"2023-07-06T20:15:18.345735+0000","last_clean":"2023-07-06T20:15:18.345735+0000","last_became_active":"2023-07-06T20:15:17.231992+0000","last_became_peered":"2023-07-06T20:15:17.231992+0000","last_unstale":"2023-07-06T20:15:18.345735+0000","last_undegraded":"2023-07-06T20:15:18.345735+0000","last_fullsized":"2023-07-06T20:15:18.345735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:02:25.230516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152742+0000","last_change":"2023-07-06T20:15:17.236809+0000","last_active":"2023-07-06T20:15:19.152742+0000","last_peered":"2023-07-06T20:15:19.152742+0000","last_clean":"2023-07-06T20:15:19.152742+0000","last_became_active":"2023-07-06T20:15:17.235497+0000","last_became_peered":"2023-07-06T20:15:17.235497+0000","last_unstale":"2023-07-06T20:15:19.152742+0000","last_undegraded":"2023-07-06T20:15:19.152742+0000","last_fullsized":"2023-07-06T20:15:19.152742+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:49:34.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345387+0000","last_change":"2023-07-06T20:15:17.240702+0000","last_active":"2023-07-06T20:15:18.345387+0000","last_peered":"2023-07-06T20:15:18.345387+0000","last_clean":"2023-07-06T20:15:18.345387+0000","last_became_active":"2023-07-06T20:15:17.240399+0000","last_became_peered":"2023-07-06T20:15:17.240399+0000","last_unstale":"2023-07-06T20:15:18.345387+0000","last_undegraded":"2023-07-06T20:15:18.345387+0000","last_fullsized":"2023-07-06T20:15:18.345387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:37.253145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453717+0000","last_change":"2023-07-06T20:15:17.242077+0000","last_active":"2023-07-06T20:15:18.453717+0000","last_peered":"2023-07-06T20:15:18.453717+0000","last_clean":"2023-07-06T20:15:18.453717+0000","last_became_active":"2023-07-06T20:15:17.241776+0000","last_became_peered":"2023-07-06T20:15:17.241776+0000","last_unstale":"2023-07-06T20:15:18.453717+0000","last_undegraded":"2023-07-06T20:15:18.453717+0000","last_fullsized":"2023-07-06T20:15:18.453717+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:34:54.294041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345154+0000","last_change":"2023-07-06T20:15:17.232316+0000","last_active":"2023-07-06T20:15:18.345154+0000","last_peered":"2023-07-06T20:15:18.345154+0000","last_clean":"2023-07-06T20:15:18.345154+0000","last_became_active":"2023-07-06T20:15:17.232069+0000","last_became_peered":"2023-07-06T20:15:17.232069+0000","last_unstale":"2023-07-06T20:15:18.345154+0000","last_undegraded":"2023-07-06T20:15:18.345154+0000","last_fullsized":"2023-07-06T20:15:18.345154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:35.832975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322254+0000","last_change":"2023-07-06T20:15:17.246820+0000","last_active":"2023-07-06T20:15:18.322254+0000","last_peered":"2023-07-06T20:15:18.322254+0000","last_clean":"2023-07-06T20:15:18.322254+0000","last_became_active":"2023-07-06T20:15:17.246540+0000","last_became_peered":"2023-07-06T20:15:17.246540+0000","last_unstale":"2023-07-06T20:15:18.322254+0000","last_undegraded":"2023-07-06T20:15:18.322254+0000","last_fullsized":"2023-07-06T20:15:18.322254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:02:45.009876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322714+0000","last_change":"2023-07-06T20:15:17.246391+0000","last_active":"2023-07-06T20:15:18.322714+0000","last_peered":"2023-07-06T20:15:18.322714+0000","last_clean":"2023-07-06T20:15:18.322714+0000","last_became_active":"2023-07-06T20:15:17.246107+0000","last_became_peered":"2023-07-06T20:15:17.246107+0000","last_unstale":"2023-07-06T20:15:18.322714+0000","last_undegraded":"2023-07-06T20:15:18.322714+0000","last_fullsized":"2023-07-06T20:15:18.322714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:54:42.425621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218412+0000","last_change":"2023-07-06T20:15:17.243903+0000","last_active":"2023-07-06T20:15:18.218412+0000","last_peered":"2023-07-06T20:15:18.218412+0000","last_clean":"2023-07-06T20:15:18.218412+0000","last_became_active":"2023-07-06T20:15:17.242739+0000","last_became_peered":"2023-07-06T20:15:17.242739+0000","last_unstale":"2023-07-06T20:15:18.218412+0000","last_undegraded":"2023-07-06T20:15:18.218412+0000","last_fullsized":"2023-07-06T20:15:18.218412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:40.235154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216880+0000","last_change":"2023-07-06T20:15:17.223730+0000","last_active":"2023-07-06T20:15:18.216880+0000","last_peered":"2023-07-06T20:15:18.216880+0000","last_clean":"2023-07-06T20:15:18.216880+0000","last_became_active":"2023-07-06T20:15:17.223377+0000","last_became_peered":"2023-07-06T20:15:17.223377+0000","last_unstale":"2023-07-06T20:15:18.216880+0000","last_undegraded":"2023-07-06T20:15:18.216880+0000","last_fullsized":"2023-07-06T20:15:18.216880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:47:53.614657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218221+0000","last_change":"2023-07-06T20:15:17.242985+0000","last_active":"2023-07-06T20:15:18.218221+0000","last_peered":"2023-07-06T20:15:18.218221+0000","last_clean":"2023-07-06T20:15:18.218221+0000","last_became_active":"2023-07-06T20:15:17.242599+0000","last_became_peered":"2023-07-06T20:15:17.242599+0000","last_unstale":"2023-07-06T20:15:18.218221+0000","last_undegraded":"2023-07-06T20:15:18.218221+0000","last_fullsized":"2023-07-06T20:15:18.218221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:25.185264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345505+0000","last_change":"2023-07-06T20:15:17.227709+0000","last_active":"2023-07-06T20:15:18.345505+0000","last_peered":"2023-07-06T20:15:18.345505+0000","last_clean":"2023-07-06T20:15:18.345505+0000","last_became_active":"2023-07-06T20:15:17.227450+0000","last_became_peered":"2023-07-06T20:15:17.227450+0000","last_unstale":"2023-07-06T20:15:18.345505+0000","last_undegraded":"2023-07-06T20:15:18.345505+0000","last_fullsized":"2023-07-06T20:15:18.345505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:02:45.009876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453278+0000","last_change":"2023-07-06T20:15:17.235436+0000","last_active":"2023-07-06T20:15:18.453278+0000","last_peered":"2023-07-06T20:15:18.453278+0000","last_clean":"2023-07-06T20:15:18.453278+0000","last_became_active":"2023-07-06T20:15:17.235038+0000","last_became_peered":"2023-07-06T20:15:17.235038+0000","last_unstale":"2023-07-06T20:15:18.453278+0000","last_undegraded":"2023-07-06T20:15:18.453278+0000","last_fullsized":"2023-07-06T20:15:18.453278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:03:44.134546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322657+0000","last_change":"2023-07-06T20:15:17.238641+0000","last_active":"2023-07-06T20:15:18.322657+0000","last_peered":"2023-07-06T20:15:18.322657+0000","last_clean":"2023-07-06T20:15:18.322657+0000","last_became_active":"2023-07-06T20:15:17.238349+0000","last_became_peered":"2023-07-06T20:15:17.238349+0000","last_unstale":"2023-07-06T20:15:18.322657+0000","last_undegraded":"2023-07-06T20:15:18.322657+0000","last_fullsized":"2023-07-06T20:15:18.322657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:07.779943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216736+0000","last_change":"2023-07-06T20:15:17.239695+0000","last_active":"2023-07-06T20:15:18.216736+0000","last_peered":"2023-07-06T20:15:18.216736+0000","last_clean":"2023-07-06T20:15:18.216736+0000","last_became_active":"2023-07-06T20:15:17.238712+0000","last_became_peered":"2023-07-06T20:15:17.238712+0000","last_unstale":"2023-07-06T20:15:18.216736+0000","last_undegraded":"2023-07-06T20:15:18.216736+0000","last_fullsized":"2023-07-06T20:15:18.216736+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:25.185264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218432+0000","last_change":"2023-07-06T20:15:17.238232+0000","last_active":"2023-07-06T20:15:18.218432+0000","last_peered":"2023-07-06T20:15:18.218432+0000","last_clean":"2023-07-06T20:15:18.218432+0000","last_became_active":"2023-07-06T20:15:17.237964+0000","last_became_peered":"2023-07-06T20:15:17.237964+0000","last_unstale":"2023-07-06T20:15:18.218432+0000","last_undegraded":"2023-07-06T20:15:18.218432+0000","last_fullsized":"2023-07-06T20:15:18.218432+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:00:22.922695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701142+0000","last_change":"2023-07-06T20:15:17.227524+0000","last_active":"2023-07-06T20:15:18.701142+0000","last_peered":"2023-07-06T20:15:18.701142+0000","last_clean":"2023-07-06T20:15:18.701142+0000","last_became_active":"2023-07-06T20:15:17.227258+0000","last_became_peered":"2023-07-06T20:15:17.227258+0000","last_unstale":"2023-07-06T20:15:18.701142+0000","last_undegraded":"2023-07-06T20:15:18.701142+0000","last_fullsized":"2023-07-06T20:15:18.701142+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:35.832975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218550+0000","last_change":"2023-07-06T20:15:17.237130+0000","last_active":"2023-07-06T20:15:18.218550+0000","last_peered":"2023-07-06T20:15:18.218550+0000","last_clean":"2023-07-06T20:15:18.218550+0000","last_became_active":"2023-07-06T20:15:17.236841+0000","last_became_peered":"2023-07-06T20:15:17.236841+0000","last_unstale":"2023-07-06T20:15:18.218550+0000","last_undegraded":"2023-07-06T20:15:18.218550+0000","last_fullsized":"2023-07-06T20:15:18.218550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:19:05.597036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345603+0000","last_change":"2023-07-06T20:15:17.224030+0000","last_active":"2023-07-06T20:15:18.345603+0000","last_peered":"2023-07-06T20:15:18.345603+0000","last_clean":"2023-07-06T20:15:18.345603+0000","last_became_active":"2023-07-06T20:15:17.223733+0000","last_became_peered":"2023-07-06T20:15:17.223733+0000","last_unstale":"2023-07-06T20:15:18.345603+0000","last_undegraded":"2023-07-06T20:15:18.345603+0000","last_fullsized":"2023-07-06T20:15:18.345603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:37:01.758729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218391+0000","last_change":"2023-07-06T20:15:17.240811+0000","last_active":"2023-07-06T20:15:18.218391+0000","last_peered":"2023-07-06T20:15:18.218391+0000","last_clean":"2023-07-06T20:15:18.218391+0000","last_became_active":"2023-07-06T20:15:17.240289+0000","last_became_peered":"2023-07-06T20:15:17.240289+0000","last_unstale":"2023-07-06T20:15:18.218391+0000","last_undegraded":"2023-07-06T20:15:18.218391+0000","last_fullsized":"2023-07-06T20:15:18.218391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:30:03.586651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218260+0000","last_change":"2023-07-06T20:15:17.243674+0000","last_active":"2023-07-06T20:15:18.218260+0000","last_peered":"2023-07-06T20:15:18.218260+0000","last_clean":"2023-07-06T20:15:18.218260+0000","last_became_active":"2023-07-06T20:15:17.243391+0000","last_became_peered":"2023-07-06T20:15:17.243391+0000","last_unstale":"2023-07-06T20:15:18.218260+0000","last_undegraded":"2023-07-06T20:15:18.218260+0000","last_fullsized":"2023-07-06T20:15:18.218260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:47:53.614657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322342+0000","last_change":"2023-07-06T20:15:17.236869+0000","last_active":"2023-07-06T20:15:18.322342+0000","last_peered":"2023-07-06T20:15:18.322342+0000","last_clean":"2023-07-06T20:15:18.322342+0000","last_became_active":"2023-07-06T20:15:17.236587+0000","last_became_peered":"2023-07-06T20:15:17.236587+0000","last_unstale":"2023-07-06T20:15:18.322342+0000","last_undegraded":"2023-07-06T20:15:18.322342+0000","last_fullsized":"2023-07-06T20:15:18.322342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:37:01.758729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700819+0000","last_change":"2023-07-06T20:15:17.222615+0000","last_active":"2023-07-06T20:15:18.700819+0000","last_peered":"2023-07-06T20:15:18.700819+0000","last_clean":"2023-07-06T20:15:18.700819+0000","last_became_active":"2023-07-06T20:15:17.222337+0000","last_became_peered":"2023-07-06T20:15:17.222337+0000","last_unstale":"2023-07-06T20:15:18.700819+0000","last_undegraded":"2023-07-06T20:15:18.700819+0000","last_fullsized":"2023-07-06T20:15:18.700819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:59.691407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322714+0000","last_change":"2023-07-06T20:15:17.245128+0000","last_active":"2023-07-06T20:15:18.322714+0000","last_peered":"2023-07-06T20:15:18.322714+0000","last_clean":"2023-07-06T20:15:18.322714+0000","last_became_active":"2023-07-06T20:15:17.244812+0000","last_became_peered":"2023-07-06T20:15:17.244812+0000","last_unstale":"2023-07-06T20:15:18.322714+0000","last_undegraded":"2023-07-06T20:15:18.322714+0000","last_fullsized":"2023-07-06T20:15:18.322714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:14:41.364559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218429+0000","last_change":"2023-07-06T20:15:16.226731+0000","last_active":"2023-07-06T20:15:18.218429+0000","last_peered":"2023-07-06T20:15:18.218429+0000","last_clean":"2023-07-06T20:15:18.218429+0000","last_became_active":"2023-07-06T20:15:16.226436+0000","last_became_peered":"2023-07-06T20:15:16.226436+0000","last_unstale":"2023-07-06T20:15:18.218429+0000","last_undegraded":"2023-07-06T20:15:18.218429+0000","last_fullsized":"2023-07-06T20:15:18.218429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:30:12.897990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701226+0000","last_change":"2023-07-06T20:15:17.226548+0000","last_active":"2023-07-06T20:15:18.701226+0000","last_peered":"2023-07-06T20:15:18.701226+0000","last_clean":"2023-07-06T20:15:18.701226+0000","last_became_active":"2023-07-06T20:15:17.226270+0000","last_became_peered":"2023-07-06T20:15:17.226270+0000","last_unstale":"2023-07-06T20:15:18.701226+0000","last_undegraded":"2023-07-06T20:15:18.701226+0000","last_fullsized":"2023-07-06T20:15:18.701226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.909334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152364+0000","last_change":"2023-07-06T20:15:16.222300+0000","last_active":"2023-07-06T20:15:19.152364+0000","last_peered":"2023-07-06T20:15:19.152364+0000","last_clean":"2023-07-06T20:15:19.152364+0000","last_became_active":"2023-07-06T20:15:16.222038+0000","last_became_peered":"2023-07-06T20:15:16.222038+0000","last_unstale":"2023-07-06T20:15:19.152364+0000","last_undegraded":"2023-07-06T20:15:19.152364+0000","last_fullsized":"2023-07-06T20:15:19.152364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:19:02.243527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453675+0000","last_change":"2023-07-06T20:15:17.240535+0000","last_active":"2023-07-06T20:15:18.453675+0000","last_peered":"2023-07-06T20:15:18.453675+0000","last_clean":"2023-07-06T20:15:18.453675+0000","last_became_active":"2023-07-06T20:15:17.240252+0000","last_became_peered":"2023-07-06T20:15:17.240252+0000","last_unstale":"2023-07-06T20:15:18.453675+0000","last_undegraded":"2023-07-06T20:15:18.453675+0000","last_fullsized":"2023-07-06T20:15:18.453675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:19.450885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701253+0000","last_change":"2023-07-06T20:15:16.219402+0000","last_active":"2023-07-06T20:15:18.701253+0000","last_peered":"2023-07-06T20:15:18.701253+0000","last_clean":"2023-07-06T20:15:18.701253+0000","last_became_active":"2023-07-06T20:15:16.219073+0000","last_became_peered":"2023-07-06T20:15:16.219073+0000","last_unstale":"2023-07-06T20:15:18.701253+0000","last_undegraded":"2023-07-06T20:15:18.701253+0000","last_fullsized":"2023-07-06T20:15:18.701253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:16.795193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152186+0000","last_change":"2023-07-06T20:15:17.238324+0000","last_active":"2023-07-06T20:15:19.152186+0000","last_peered":"2023-07-06T20:15:19.152186+0000","last_clean":"2023-07-06T20:15:19.152186+0000","last_became_active":"2023-07-06T20:15:17.238109+0000","last_became_peered":"2023-07-06T20:15:17.238109+0000","last_unstale":"2023-07-06T20:15:19.152186+0000","last_undegraded":"2023-07-06T20:15:19.152186+0000","last_fullsized":"2023-07-06T20:15:19.152186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:30.023080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453479+0000","last_change":"2023-07-06T20:15:16.217117+0000","last_active":"2023-07-06T20:15:18.453479+0000","last_peered":"2023-07-06T20:15:18.453479+0000","last_clean":"2023-07-06T20:15:18.453479+0000","last_became_active":"2023-07-06T20:15:16.214703+0000","last_became_peered":"2023-07-06T20:15:16.214703+0000","last_unstale":"2023-07-06T20:15:18.453479+0000","last_undegraded":"2023-07-06T20:15:18.453479+0000","last_fullsized":"2023-07-06T20:15:18.453479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:28:59.869660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152238+0000","last_change":"2023-07-06T20:15:17.241995+0000","last_active":"2023-07-06T20:15:19.152238+0000","last_peered":"2023-07-06T20:15:19.152238+0000","last_clean":"2023-07-06T20:15:19.152238+0000","last_became_active":"2023-07-06T20:15:17.241705+0000","last_became_peered":"2023-07-06T20:15:17.241705+0000","last_unstale":"2023-07-06T20:15:19.152238+0000","last_undegraded":"2023-07-06T20:15:19.152238+0000","last_fullsized":"2023-07-06T20:15:19.152238+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:27:58.742569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345751+0000","last_change":"2023-07-06T20:15:16.220418+0000","last_active":"2023-07-06T20:15:18.345751+0000","last_peered":"2023-07-06T20:15:18.345751+0000","last_clean":"2023-07-06T20:15:18.345751+0000","last_became_active":"2023-07-06T20:15:16.220049+0000","last_became_peered":"2023-07-06T20:15:16.220049+0000","last_unstale":"2023-07-06T20:15:18.345751+0000","last_undegraded":"2023-07-06T20:15:18.345751+0000","last_fullsized":"2023-07-06T20:15:18.345751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:04.894008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345619+0000","last_change":"2023-07-06T20:15:17.231816+0000","last_active":"2023-07-06T20:15:18.345619+0000","last_peered":"2023-07-06T20:15:18.345619+0000","last_clean":"2023-07-06T20:15:18.345619+0000","last_became_active":"2023-07-06T20:15:17.231525+0000","last_became_peered":"2023-07-06T20:15:17.231525+0000","last_unstale":"2023-07-06T20:15:18.345619+0000","last_undegraded":"2023-07-06T20:15:18.345619+0000","last_fullsized":"2023-07-06T20:15:18.345619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:17.802296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322342+0000","last_change":"2023-07-06T20:15:16.226735+0000","last_active":"2023-07-06T20:15:18.322342+0000","last_peered":"2023-07-06T20:15:18.322342+0000","last_clean":"2023-07-06T20:15:18.322342+0000","last_became_active":"2023-07-06T20:15:16.226454+0000","last_became_peered":"2023-07-06T20:15:16.226454+0000","last_unstale":"2023-07-06T20:15:18.322342+0000","last_undegraded":"2023-07-06T20:15:18.322342+0000","last_fullsized":"2023-07-06T20:15:18.322342+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:42.703860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453277+0000","last_change":"2023-07-06T20:15:17.235861+0000","last_active":"2023-07-06T20:15:18.453277+0000","last_peered":"2023-07-06T20:15:18.453277+0000","last_clean":"2023-07-06T20:15:18.453277+0000","last_became_active":"2023-07-06T20:15:17.235525+0000","last_became_peered":"2023-07-06T20:15:17.235525+0000","last_unstale":"2023-07-06T20:15:18.453277+0000","last_undegraded":"2023-07-06T20:15:18.453277+0000","last_fullsized":"2023-07-06T20:15:18.453277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:00.880380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700939+0000","last_change":"2023-07-06T20:15:16.221347+0000","last_active":"2023-07-06T20:15:18.700939+0000","last_peered":"2023-07-06T20:15:18.700939+0000","last_clean":"2023-07-06T20:15:18.700939+0000","last_became_active":"2023-07-06T20:15:16.221064+0000","last_became_peered":"2023-07-06T20:15:16.221064+0000","last_unstale":"2023-07-06T20:15:18.700939+0000","last_undegraded":"2023-07-06T20:15:18.700939+0000","last_fullsized":"2023-07-06T20:15:18.700939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:26.287448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152349+0000","last_change":"2023-07-06T20:15:17.243169+0000","last_active":"2023-07-06T20:15:19.152349+0000","last_peered":"2023-07-06T20:15:19.152349+0000","last_clean":"2023-07-06T20:15:19.152349+0000","last_became_active":"2023-07-06T20:15:17.242838+0000","last_became_peered":"2023-07-06T20:15:17.242838+0000","last_unstale":"2023-07-06T20:15:19.152349+0000","last_undegraded":"2023-07-06T20:15:19.152349+0000","last_fullsized":"2023-07-06T20:15:19.152349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:17.802296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453240+0000","last_change":"2023-07-06T20:15:16.221133+0000","last_active":"2023-07-06T20:15:18.453240+0000","last_peered":"2023-07-06T20:15:18.453240+0000","last_clean":"2023-07-06T20:15:18.453240+0000","last_became_active":"2023-07-06T20:15:16.220855+0000","last_became_peered":"2023-07-06T20:15:16.220855+0000","last_unstale":"2023-07-06T20:15:18.453240+0000","last_undegraded":"2023-07-06T20:15:18.453240+0000","last_fullsized":"2023-07-06T20:15:18.453240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:18:27.414965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.217000+0000","last_change":"2023-07-06T20:15:17.245308+0000","last_active":"2023-07-06T20:15:18.217000+0000","last_peered":"2023-07-06T20:15:18.217000+0000","last_clean":"2023-07-06T20:15:18.217000+0000","last_became_active":"2023-07-06T20:15:17.245040+0000","last_became_peered":"2023-07-06T20:15:17.245040+0000","last_unstale":"2023-07-06T20:15:18.217000+0000","last_undegraded":"2023-07-06T20:15:18.217000+0000","last_fullsized":"2023-07-06T20:15:18.217000+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:40.235154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152265+0000","last_change":"2023-07-06T20:15:16.210293+0000","last_active":"2023-07-06T20:15:19.152265+0000","last_peered":"2023-07-06T20:15:19.152265+0000","last_clean":"2023-07-06T20:15:19.152265+0000","last_became_active":"2023-07-06T20:15:16.209994+0000","last_became_peered":"2023-07-06T20:15:16.209994+0000","last_unstale":"2023-07-06T20:15:19.152265+0000","last_undegraded":"2023-07-06T20:15:19.152265+0000","last_fullsized":"2023-07-06T20:15:19.152265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:12.902230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322584+0000","last_change":"2023-07-06T20:15:17.241717+0000","last_active":"2023-07-06T20:15:18.322584+0000","last_peered":"2023-07-06T20:15:18.322584+0000","last_clean":"2023-07-06T20:15:18.322584+0000","last_became_active":"2023-07-06T20:15:17.241455+0000","last_became_peered":"2023-07-06T20:15:17.241455+0000","last_unstale":"2023-07-06T20:15:18.322584+0000","last_undegraded":"2023-07-06T20:15:18.322584+0000","last_fullsized":"2023-07-06T20:15:18.322584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:05.901112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345928+0000","last_change":"2023-07-06T20:15:16.216548+0000","last_active":"2023-07-06T20:15:18.345928+0000","last_peered":"2023-07-06T20:15:18.345928+0000","last_clean":"2023-07-06T20:15:18.345928+0000","last_became_active":"2023-07-06T20:15:16.215145+0000","last_became_peered":"2023-07-06T20:15:16.215145+0000","last_unstale":"2023-07-06T20:15:18.345928+0000","last_undegraded":"2023-07-06T20:15:18.345928+0000","last_fullsized":"2023-07-06T20:15:18.345928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:06.772839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152211+0000","last_change":"2023-07-06T20:15:17.237487+0000","last_active":"2023-07-06T20:15:19.152211+0000","last_peered":"2023-07-06T20:15:19.152211+0000","last_clean":"2023-07-06T20:15:19.152211+0000","last_became_active":"2023-07-06T20:15:17.237204+0000","last_became_peered":"2023-07-06T20:15:17.237204+0000","last_unstale":"2023-07-06T20:15:19.152211+0000","last_undegraded":"2023-07-06T20:15:19.152211+0000","last_fullsized":"2023-07-06T20:15:19.152211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:13.846737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345831+0000","last_change":"2023-07-06T20:15:16.218752+0000","last_active":"2023-07-06T20:15:18.345831+0000","last_peered":"2023-07-06T20:15:18.345831+0000","last_clean":"2023-07-06T20:15:18.345831+0000","last_became_active":"2023-07-06T20:15:16.218262+0000","last_became_peered":"2023-07-06T20:15:16.218262+0000","last_unstale":"2023-07-06T20:15:18.345831+0000","last_undegraded":"2023-07-06T20:15:18.345831+0000","last_fullsized":"2023-07-06T20:15:18.345831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:14:40.357455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453842+0000","last_change":"2023-07-06T20:15:17.235525+0000","last_active":"2023-07-06T20:15:18.453842+0000","last_peered":"2023-07-06T20:15:18.453842+0000","last_clean":"2023-07-06T20:15:18.453842+0000","last_became_active":"2023-07-06T20:15:17.235191+0000","last_became_peered":"2023-07-06T20:15:17.235191+0000","last_unstale":"2023-07-06T20:15:18.453842+0000","last_undegraded":"2023-07-06T20:15:18.453842+0000","last_fullsized":"2023-07-06T20:15:18.453842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:40:36.235206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152580+0000","last_change":"2023-07-06T20:15:16.220025+0000","last_active":"2023-07-06T20:15:19.152580+0000","last_peered":"2023-07-06T20:15:19.152580+0000","last_clean":"2023-07-06T20:15:19.152580+0000","last_became_active":"2023-07-06T20:15:16.219733+0000","last_became_peered":"2023-07-06T20:15:16.219733+0000","last_unstale":"2023-07-06T20:15:19.152580+0000","last_undegraded":"2023-07-06T20:15:19.152580+0000","last_fullsized":"2023-07-06T20:15:19.152580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:34:48.403559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152469+0000","last_change":"2023-07-06T20:15:17.246117+0000","last_active":"2023-07-06T20:15:19.152469+0000","last_peered":"2023-07-06T20:15:19.152469+0000","last_clean":"2023-07-06T20:15:19.152469+0000","last_became_active":"2023-07-06T20:15:17.245802+0000","last_became_peered":"2023-07-06T20:15:17.245802+0000","last_unstale":"2023-07-06T20:15:19.152469+0000","last_undegraded":"2023-07-06T20:15:19.152469+0000","last_fullsized":"2023-07-06T20:15:19.152469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:48:52.572201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152475+0000","last_change":"2023-07-06T20:15:16.217580+0000","last_active":"2023-07-06T20:15:19.152475+0000","last_peered":"2023-07-06T20:15:19.152475+0000","last_clean":"2023-07-06T20:15:19.152475+0000","last_became_active":"2023-07-06T20:15:16.217319+0000","last_became_peered":"2023-07-06T20:15:16.217319+0000","last_unstale":"2023-07-06T20:15:19.152475+0000","last_undegraded":"2023-07-06T20:15:19.152475+0000","last_fullsized":"2023-07-06T20:15:19.152475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:02:44.002773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152031+0000","last_change":"2023-07-06T20:15:17.233452+0000","last_active":"2023-07-06T20:15:19.152031+0000","last_peered":"2023-07-06T20:15:19.152031+0000","last_clean":"2023-07-06T20:15:19.152031+0000","last_became_active":"2023-07-06T20:15:17.233213+0000","last_became_peered":"2023-07-06T20:15:17.233213+0000","last_unstale":"2023-07-06T20:15:19.152031+0000","last_undegraded":"2023-07-06T20:15:19.152031+0000","last_fullsized":"2023-07-06T20:15:19.152031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:13:42.344623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700993+0000","last_change":"2023-07-06T20:15:16.221151+0000","last_active":"2023-07-06T20:15:18.700993+0000","last_peered":"2023-07-06T20:15:18.700993+0000","last_clean":"2023-07-06T20:15:18.700993+0000","last_became_active":"2023-07-06T20:15:16.220895+0000","last_became_peered":"2023-07-06T20:15:16.220895+0000","last_unstale":"2023-07-06T20:15:18.700993+0000","last_undegraded":"2023-07-06T20:15:18.700993+0000","last_fullsized":"2023-07-06T20:15:18.700993+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:48.762899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322646+0000","last_change":"2023-07-06T20:15:17.238985+0000","last_active":"2023-07-06T20:15:18.322646+0000","last_peered":"2023-07-06T20:15:18.322646+0000","last_clean":"2023-07-06T20:15:18.322646+0000","last_became_active":"2023-07-06T20:15:17.238701+0000","last_became_peered":"2023-07-06T20:15:17.238701+0000","last_unstale":"2023-07-06T20:15:18.322646+0000","last_undegraded":"2023-07-06T20:15:18.322646+0000","last_fullsized":"2023-07-06T20:15:18.322646+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:10.099381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.217961+0000","last_change":"2023-07-06T20:15:16.226686+0000","last_active":"2023-07-06T20:15:18.217961+0000","last_peered":"2023-07-06T20:15:18.217961+0000","last_clean":"2023-07-06T20:15:18.217961+0000","last_became_active":"2023-07-06T20:15:16.226395+0000","last_became_peered":"2023-07-06T20:15:16.226395+0000","last_unstale":"2023-07-06T20:15:18.217961+0000","last_undegraded":"2023-07-06T20:15:18.217961+0000","last_fullsized":"2023-07-06T20:15:18.217961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:59.015168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152012+0000","last_change":"2023-07-06T20:15:17.237561+0000","last_active":"2023-07-06T20:15:19.152012+0000","last_peered":"2023-07-06T20:15:19.152012+0000","last_clean":"2023-07-06T20:15:19.152012+0000","last_became_active":"2023-07-06T20:15:17.237249+0000","last_became_peered":"2023-07-06T20:15:17.237249+0000","last_unstale":"2023-07-06T20:15:19.152012+0000","last_undegraded":"2023-07-06T20:15:19.152012+0000","last_fullsized":"2023-07-06T20:15:19.152012+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:49.770003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322439+0000","last_change":"2023-07-06T20:15:16.222284+0000","last_active":"2023-07-06T20:15:18.322439+0000","last_peered":"2023-07-06T20:15:18.322439+0000","last_clean":"2023-07-06T20:15:18.322439+0000","last_became_active":"2023-07-06T20:15:16.222004+0000","last_became_peered":"2023-07-06T20:15:16.222004+0000","last_unstale":"2023-07-06T20:15:18.322439+0000","last_undegraded":"2023-07-06T20:15:18.322439+0000","last_fullsized":"2023-07-06T20:15:18.322439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:02:24.223412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152527+0000","last_change":"2023-07-06T20:15:17.240664+0000","last_active":"2023-07-06T20:15:19.152527+0000","last_peered":"2023-07-06T20:15:19.152527+0000","last_clean":"2023-07-06T20:15:19.152527+0000","last_became_active":"2023-07-06T20:15:17.240366+0000","last_became_peered":"2023-07-06T20:15:17.240366+0000","last_unstale":"2023-07-06T20:15:19.152527+0000","last_undegraded":"2023-07-06T20:15:19.152527+0000","last_fullsized":"2023-07-06T20:15:19.152527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:10.099381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.346073+0000","last_change":"2023-07-06T20:15:16.224694+0000","last_active":"2023-07-06T20:15:18.346073+0000","last_peered":"2023-07-06T20:15:18.346073+0000","last_clean":"2023-07-06T20:15:18.346073+0000","last_became_active":"2023-07-06T20:15:16.224368+0000","last_became_peered":"2023-07-06T20:15:16.224368+0000","last_unstale":"2023-07-06T20:15:18.346073+0000","last_undegraded":"2023-07-06T20:15:18.346073+0000","last_fullsized":"2023-07-06T20:15:18.346073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:54:41.418518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345667+0000","last_change":"2023-07-06T20:15:17.231710+0000","last_active":"2023-07-06T20:15:18.345667+0000","last_peered":"2023-07-06T20:15:18.345667+0000","last_clean":"2023-07-06T20:15:18.345667+0000","last_became_active":"2023-07-06T20:15:17.229055+0000","last_became_peered":"2023-07-06T20:15:17.229055+0000","last_unstale":"2023-07-06T20:15:18.345667+0000","last_undegraded":"2023-07-06T20:15:18.345667+0000","last_fullsized":"2023-07-06T20:15:18.345667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:43.710964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453814+0000","last_change":"2023-07-06T20:15:16.219291+0000","last_active":"2023-07-06T20:15:18.453814+0000","last_peered":"2023-07-06T20:15:18.453814+0000","last_clean":"2023-07-06T20:15:18.453814+0000","last_became_active":"2023-07-06T20:15:16.218872+0000","last_became_peered":"2023-07-06T20:15:16.218872+0000","last_unstale":"2023-07-06T20:15:18.453814+0000","last_undegraded":"2023-07-06T20:15:18.453814+0000","last_fullsized":"2023-07-06T20:15:18.453814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:40.992338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453699+0000","last_change":"2023-07-06T20:15:17.237273+0000","last_active":"2023-07-06T20:15:18.453699+0000","last_peered":"2023-07-06T20:15:18.453699+0000","last_clean":"2023-07-06T20:15:18.453699+0000","last_became_active":"2023-07-06T20:15:17.236996+0000","last_became_peered":"2023-07-06T20:15:17.236996+0000","last_unstale":"2023-07-06T20:15:18.453699+0000","last_undegraded":"2023-07-06T20:15:18.453699+0000","last_fullsized":"2023-07-06T20:15:18.453699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:53:52.736942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216675+0000","last_change":"2023-07-06T20:15:16.219836+0000","last_active":"2023-07-06T20:15:18.216675+0000","last_peered":"2023-07-06T20:15:18.216675+0000","last_clean":"2023-07-06T20:15:18.216675+0000","last_became_active":"2023-07-06T20:15:16.219564+0000","last_became_peered":"2023-07-06T20:15:16.219564+0000","last_unstale":"2023-07-06T20:15:18.216675+0000","last_undegraded":"2023-07-06T20:15:18.216675+0000","last_fullsized":"2023-07-06T20:15:18.216675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:44:28.034228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216997+0000","last_change":"2023-07-06T20:15:17.225239+0000","last_active":"2023-07-06T20:15:18.216997+0000","last_peered":"2023-07-06T20:15:18.216997+0000","last_clean":"2023-07-06T20:15:18.216997+0000","last_became_active":"2023-07-06T20:15:17.224941+0000","last_became_peered":"2023-07-06T20:15:17.224941+0000","last_unstale":"2023-07-06T20:15:18.216997+0000","last_undegraded":"2023-07-06T20:15:18.216997+0000","last_fullsized":"2023-07-06T20:15:18.216997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:30:03.586651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701179+0000","last_change":"2023-07-06T20:15:16.219300+0000","last_active":"2023-07-06T20:15:18.701179+0000","last_peered":"2023-07-06T20:15:18.701179+0000","last_clean":"2023-07-06T20:15:18.701179+0000","last_became_active":"2023-07-06T20:15:16.218906+0000","last_became_peered":"2023-07-06T20:15:16.218906+0000","last_unstale":"2023-07-06T20:15:18.701179+0000","last_undegraded":"2023-07-06T20:15:18.701179+0000","last_fullsized":"2023-07-06T20:15:18.701179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:36.246041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322441+0000","last_change":"2023-07-06T20:15:17.236062+0000","last_active":"2023-07-06T20:15:18.322441+0000","last_peered":"2023-07-06T20:15:18.322441+0000","last_clean":"2023-07-06T20:15:18.322441+0000","last_became_active":"2023-07-06T20:15:17.235767+0000","last_became_peered":"2023-07-06T20:15:17.235767+0000","last_unstale":"2023-07-06T20:15:18.322441+0000","last_undegraded":"2023-07-06T20:15:18.322441+0000","last_fullsized":"2023-07-06T20:15:18.322441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:34:49.410663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216730+0000","last_change":"2023-07-06T20:15:16.217727+0000","last_active":"2023-07-06T20:15:18.216730+0000","last_peered":"2023-07-06T20:15:18.216730+0000","last_clean":"2023-07-06T20:15:18.216730+0000","last_became_active":"2023-07-06T20:15:16.217389+0000","last_became_peered":"2023-07-06T20:15:16.217389+0000","last_unstale":"2023-07-06T20:15:18.216730+0000","last_undegraded":"2023-07-06T20:15:18.216730+0000","last_fullsized":"2023-07-06T20:15:18.216730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:59.015168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152286+0000","last_change":"2023-07-06T20:15:17.237275+0000","last_active":"2023-07-06T20:15:19.152286+0000","last_peered":"2023-07-06T20:15:19.152286+0000","last_clean":"2023-07-06T20:15:19.152286+0000","last_became_active":"2023-07-06T20:15:17.236862+0000","last_became_peered":"2023-07-06T20:15:17.236862+0000","last_unstale":"2023-07-06T20:15:19.152286+0000","last_undegraded":"2023-07-06T20:15:19.152286+0000","last_fullsized":"2023-07-06T20:15:19.152286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:19:18.047476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216577+0000","last_change":"2023-07-06T20:15:16.218252+0000","last_active":"2023-07-06T20:15:18.216577+0000","last_peered":"2023-07-06T20:15:18.216577+0000","last_clean":"2023-07-06T20:15:18.216577+0000","last_became_active":"2023-07-06T20:15:16.217986+0000","last_became_peered":"2023-07-06T20:15:16.217986+0000","last_unstale":"2023-07-06T20:15:18.216577+0000","last_undegraded":"2023-07-06T20:15:18.216577+0000","last_fullsized":"2023-07-06T20:15:18.216577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:02:56.259164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345352+0000","last_change":"2023-07-06T20:15:17.238880+0000","last_active":"2023-07-06T20:15:18.345352+0000","last_peered":"2023-07-06T20:15:18.345352+0000","last_clean":"2023-07-06T20:15:18.345352+0000","last_became_active":"2023-07-06T20:15:17.238661+0000","last_became_peered":"2023-07-06T20:15:17.238661+0000","last_unstale":"2023-07-06T20:15:18.345352+0000","last_undegraded":"2023-07-06T20:15:18.345352+0000","last_fullsized":"2023-07-06T20:15:18.345352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:30.023080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322862+0000","last_change":"2023-07-06T20:15:16.222750+0000","last_active":"2023-07-06T20:15:18.322862+0000","last_peered":"2023-07-06T20:15:18.322862+0000","last_clean":"2023-07-06T20:15:18.322862+0000","last_became_active":"2023-07-06T20:15:16.222400+0000","last_became_peered":"2023-07-06T20:15:16.222400+0000","last_unstale":"2023-07-06T20:15:18.322862+0000","last_undegraded":"2023-07-06T20:15:18.322862+0000","last_fullsized":"2023-07-06T20:15:18.322862+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:28:59.869660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152191+0000","last_change":"2023-07-06T20:15:17.237349+0000","last_active":"2023-07-06T20:15:19.152191+0000","last_peered":"2023-07-06T20:15:19.152191+0000","last_clean":"2023-07-06T20:15:19.152191+0000","last_became_active":"2023-07-06T20:15:17.236856+0000","last_became_peered":"2023-07-06T20:15:17.236856+0000","last_unstale":"2023-07-06T20:15:19.152191+0000","last_undegraded":"2023-07-06T20:15:19.152191+0000","last_fullsized":"2023-07-06T20:15:19.152191+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:37.253145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216715+0000","last_change":"2023-07-06T20:15:16.217890+0000","last_active":"2023-07-06T20:15:18.216715+0000","last_peered":"2023-07-06T20:15:18.216715+0000","last_clean":"2023-07-06T20:15:18.216715+0000","last_became_active":"2023-07-06T20:15:16.217589+0000","last_became_peered":"2023-07-06T20:15:16.217589+0000","last_unstale":"2023-07-06T20:15:18.216715+0000","last_undegraded":"2023-07-06T20:15:18.216715+0000","last_fullsized":"2023-07-06T20:15:18.216715+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:45:33.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152776+0000","last_change":"2023-07-06T20:15:17.244334+0000","last_active":"2023-07-06T20:15:19.152776+0000","last_peered":"2023-07-06T20:15:19.152776+0000","last_clean":"2023-07-06T20:15:19.152776+0000","last_became_active":"2023-07-06T20:15:17.244066+0000","last_became_peered":"2023-07-06T20:15:17.244066+0000","last_unstale":"2023-07-06T20:15:19.152776+0000","last_undegraded":"2023-07-06T20:15:19.152776+0000","last_fullsized":"2023-07-06T20:15:19.152776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:54:50.838801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152761+0000","last_change":"2023-07-06T20:15:16.222467+0000","last_active":"2023-07-06T20:15:19.152761+0000","last_peered":"2023-07-06T20:15:19.152761+0000","last_clean":"2023-07-06T20:15:19.152761+0000","last_became_active":"2023-07-06T20:15:16.222147+0000","last_became_peered":"2023-07-06T20:15:16.222147+0000","last_unstale":"2023-07-06T20:15:19.152761+0000","last_undegraded":"2023-07-06T20:15:19.152761+0000","last_fullsized":"2023-07-06T20:15:19.152761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:53:39.180721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701264+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:15:18.701264+0000","last_peered":"2023-07-06T20:15:18.701264+0000","last_clean":"2023-07-06T20:15:18.701264+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:18.701264+0000","last_undegraded":"2023-07-06T20:15:18.701264+0000","last_fullsized":"2023-07-06T20:15:18.701264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:01:20.771894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.151923+0000","last_change":"2023-07-06T20:15:17.233005+0000","last_active":"2023-07-06T20:15:19.151923+0000","last_peered":"2023-07-06T20:15:19.151923+0000","last_clean":"2023-07-06T20:15:19.151923+0000","last_became_active":"2023-07-06T20:15:17.232718+0000","last_became_peered":"2023-07-06T20:15:17.232718+0000","last_unstale":"2023-07-06T20:15:19.151923+0000","last_undegraded":"2023-07-06T20:15:19.151923+0000","last_fullsized":"2023-07-06T20:15:19.151923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:51:28.509267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152806+0000","last_change":"2023-07-06T20:15:16.214400+0000","last_active":"2023-07-06T20:15:19.152806+0000","last_peered":"2023-07-06T20:15:19.152806+0000","last_clean":"2023-07-06T20:15:19.152806+0000","last_became_active":"2023-07-06T20:15:16.213976+0000","last_became_peered":"2023-07-06T20:15:16.213976+0000","last_unstale":"2023-07-06T20:15:19.152806+0000","last_undegraded":"2023-07-06T20:15:19.152806+0000","last_fullsized":"2023-07-06T20:15:19.152806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:08:33.756544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345545+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:15:18.345545+0000","last_peered":"2023-07-06T20:15:18.345545+0000","last_clean":"2023-07-06T20:15:18.345545+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:18.345545+0000","last_undegraded":"2023-07-06T20:15:18.345545+0000","last_fullsized":"2023-07-06T20:15:18.345545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:25.172680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322885+0000","last_change":"2023-07-06T20:15:17.240556+0000","last_active":"2023-07-06T20:15:18.322885+0000","last_peered":"2023-07-06T20:15:18.322885+0000","last_clean":"2023-07-06T20:15:18.322885+0000","last_became_active":"2023-07-06T20:15:17.240217+0000","last_became_peered":"2023-07-06T20:15:17.240217+0000","last_unstale":"2023-07-06T20:15:18.322885+0000","last_undegraded":"2023-07-06T20:15:18.322885+0000","last_fullsized":"2023-07-06T20:15:18.322885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:19.450885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.344485+0000","last_change":"2023-07-06T20:15:16.218991+0000","last_active":"2023-07-06T20:15:18.344485+0000","last_peered":"2023-07-06T20:15:18.344485+0000","last_clean":"2023-07-06T20:15:18.344485+0000","last_became_active":"2023-07-06T20:15:16.218677+0000","last_became_peered":"2023-07-06T20:15:16.218677+0000","last_unstale":"2023-07-06T20:15:18.344485+0000","last_undegraded":"2023-07-06T20:15:18.344485+0000","last_fullsized":"2023-07-06T20:15:18.344485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:22.712178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216473+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:15:18.216473+0000","last_peered":"2023-07-06T20:15:18.216473+0000","last_clean":"2023-07-06T20:15:18.216473+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:18.216473+0000","last_undegraded":"2023-07-06T20:15:18.216473+0000","last_fullsized":"2023-07-06T20:15:18.216473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:35:21.741785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345986+0000","last_change":"2023-07-06T20:15:17.233295+0000","last_active":"2023-07-06T20:15:18.345986+0000","last_peered":"2023-07-06T20:15:18.345986+0000","last_clean":"2023-07-06T20:15:18.345986+0000","last_became_active":"2023-07-06T20:15:17.229758+0000","last_became_peered":"2023-07-06T20:15:17.229758+0000","last_unstale":"2023-07-06T20:15:18.345986+0000","last_undegraded":"2023-07-06T20:15:18.345986+0000","last_fullsized":"2023-07-06T20:15:18.345986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:18:28.422068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322773+0000","last_change":"2023-07-06T20:15:16.226751+0000","last_active":"2023-07-06T20:15:18.322773+0000","last_peered":"2023-07-06T20:15:18.322773+0000","last_clean":"2023-07-06T20:15:18.322773+0000","last_became_active":"2023-07-06T20:15:16.226508+0000","last_became_peered":"2023-07-06T20:15:16.226508+0000","last_unstale":"2023-07-06T20:15:18.322773+0000","last_undegraded":"2023-07-06T20:15:18.322773+0000","last_fullsized":"2023-07-06T20:15:18.322773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:03:43.127442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345491+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:15:18.345491+0000","last_peered":"2023-07-06T20:15:18.345491+0000","last_clean":"2023-07-06T20:15:18.345491+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:18.345491+0000","last_undegraded":"2023-07-06T20:15:18.345491+0000","last_fullsized":"2023-07-06T20:15:18.345491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:30:45.861398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322598+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:15:18.322598+0000","last_peered":"2023-07-06T20:15:18.322598+0000","last_clean":"2023-07-06T20:15:18.322598+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:18.322598+0000","last_undegraded":"2023-07-06T20:15:18.322598+0000","last_fullsized":"2023-07-06T20:15:18.322598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:28:36.054410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701164+0000","last_change":"2023-07-06T20:15:17.227424+0000","last_active":"2023-07-06T20:15:18.701164+0000","last_peered":"2023-07-06T20:15:18.701164+0000","last_clean":"2023-07-06T20:15:18.701164+0000","last_became_active":"2023-07-06T20:15:17.227133+0000","last_became_peered":"2023-07-06T20:15:17.227133+0000","last_unstale":"2023-07-06T20:15:18.701164+0000","last_undegraded":"2023-07-06T20:15:18.701164+0000","last_fullsized":"2023-07-06T20:15:18.701164+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:30.023080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.346074+0000","last_change":"2023-07-06T20:15:16.219749+0000","last_active":"2023-07-06T20:15:18.346074+0000","last_peered":"2023-07-06T20:15:18.346074+0000","last_clean":"2023-07-06T20:15:18.346074+0000","last_became_active":"2023-07-06T20:15:16.219092+0000","last_became_peered":"2023-07-06T20:15:16.219092+0000","last_unstale":"2023-07-06T20:15:18.346074+0000","last_undegraded":"2023-07-06T20:15:18.346074+0000","last_fullsized":"2023-07-06T20:15:18.346074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:03:43.127442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322644+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:15:18.322644+0000","last_peered":"2023-07-06T20:15:18.322644+0000","last_clean":"2023-07-06T20:15:18.322644+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:15:18.322644+0000","last_undegraded":"2023-07-06T20:15:18.322644+0000","last_fullsized":"2023-07-06T20:15:18.322644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:17:04.184086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700843+0000","last_change":"2023-07-06T20:15:17.240283+0000","last_active":"2023-07-06T20:15:18.700843+0000","last_peered":"2023-07-06T20:15:18.700843+0000","last_clean":"2023-07-06T20:15:18.700843+0000","last_became_active":"2023-07-06T20:15:17.239921+0000","last_became_peered":"2023-07-06T20:15:17.239921+0000","last_unstale":"2023-07-06T20:15:18.700843+0000","last_undegraded":"2023-07-06T20:15:18.700843+0000","last_fullsized":"2023-07-06T20:15:18.700843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:28:39.303552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218334+0000","last_change":"2023-07-06T20:15:16.221737+0000","last_active":"2023-07-06T20:15:18.218334+0000","last_peered":"2023-07-06T20:15:18.218334+0000","last_clean":"2023-07-06T20:15:18.218334+0000","last_became_active":"2023-07-06T20:15:16.221448+0000","last_became_peered":"2023-07-06T20:15:16.221448+0000","last_unstale":"2023-07-06T20:15:18.218334+0000","last_undegraded":"2023-07-06T20:15:18.218334+0000","last_fullsized":"2023-07-06T20:15:18.218334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:49:19.397506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322837+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:15:18.322837+0000","last_peered":"2023-07-06T20:15:18.322837+0000","last_clean":"2023-07-06T20:15:18.322837+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:18.322837+0000","last_undegraded":"2023-07-06T20:15:18.322837+0000","last_fullsized":"2023-07-06T20:15:18.322837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:40:43.891245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453499+0000","last_change":"2023-07-06T20:15:17.228706+0000","last_active":"2023-07-06T20:15:18.453499+0000","last_peered":"2023-07-06T20:15:18.453499+0000","last_clean":"2023-07-06T20:15:18.453499+0000","last_became_active":"2023-07-06T20:15:17.228435+0000","last_became_peered":"2023-07-06T20:15:17.228435+0000","last_unstale":"2023-07-06T20:15:18.453499+0000","last_undegraded":"2023-07-06T20:15:18.453499+0000","last_fullsized":"2023-07-06T20:15:18.453499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:42:08.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453252+0000","last_change":"2023-07-06T20:15:16.214042+0000","last_active":"2023-07-06T20:15:18.453252+0000","last_peered":"2023-07-06T20:15:18.453252+0000","last_clean":"2023-07-06T20:15:18.453252+0000","last_became_active":"2023-07-06T20:15:16.213603+0000","last_became_peered":"2023-07-06T20:15:16.213603+0000","last_unstale":"2023-07-06T20:15:18.453252+0000","last_undegraded":"2023-07-06T20:15:18.453252+0000","last_fullsized":"2023-07-06T20:15:18.453252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:54:41.418518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701081+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:15:18.701081+0000","last_peered":"2023-07-06T20:15:18.701081+0000","last_clean":"2023-07-06T20:15:18.701081+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:18.701081+0000","last_undegraded":"2023-07-06T20:15:18.701081+0000","last_fullsized":"2023-07-06T20:15:18.701081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:12:18.106641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322790+0000","last_change":"2023-07-06T20:15:17.243359+0000","last_active":"2023-07-06T20:15:18.322790+0000","last_peered":"2023-07-06T20:15:18.322790+0000","last_clean":"2023-07-06T20:15:18.322790+0000","last_became_active":"2023-07-06T20:15:17.243081+0000","last_became_peered":"2023-07-06T20:15:17.243081+0000","last_unstale":"2023-07-06T20:15:18.322790+0000","last_undegraded":"2023-07-06T20:15:18.322790+0000","last_fullsized":"2023-07-06T20:15:18.322790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:00.880380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.151791+0000","last_change":"2023-07-06T20:15:16.210001+0000","last_active":"2023-07-06T20:15:19.151791+0000","last_peered":"2023-07-06T20:15:19.151791+0000","last_clean":"2023-07-06T20:15:19.151791+0000","last_became_active":"2023-07-06T20:15:16.209479+0000","last_became_peered":"2023-07-06T20:15:16.209479+0000","last_unstale":"2023-07-06T20:15:19.151791+0000","last_undegraded":"2023-07-06T20:15:19.151791+0000","last_fullsized":"2023-07-06T20:15:19.151791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:28:38.296448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218535+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:15:18.218535+0000","last_peered":"2023-07-06T20:15:18.218535+0000","last_clean":"2023-07-06T20:15:18.218535+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:18.218535+0000","last_undegraded":"2023-07-06T20:15:18.218535+0000","last_fullsized":"2023-07-06T20:15:18.218535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:44.051392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":761722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.81499999999999995}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.64600000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.875}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":170302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.629}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.503}]},{"osd":7,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.307},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.875}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.746}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":752450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.876},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.048}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.089}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":164665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":11.109999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.324999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":11.145,"5min":11.145,"15min":11.145},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":11.664,"5min":11.664,"15min":11.664},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":11.208,"5min":11.208,"15min":11.208},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.997999999999999,"5min":10.997999999999999,"15min":10.997999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":11.286,"5min":11.286,"15min":11.286},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.876},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":11.055,"5min":11.055,"15min":11.055},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":11.247999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":170324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.0010000000000003,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.7939999999999996,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.75},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":5.1710000000000003,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.84799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":169832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.624}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:15:21.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:15:21.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.906+0000 7fbf8edff700 1 Processor -- start 2023-07-06T20:15:21.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.906+0000 7fbf8edff700 1 -- start start 2023-07-06T20:15:21.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.907+0000 7fbf8edff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbf8815fca0 0x7fbf880d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.907+0000 7fbf8edff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf880cf780 con 0x7fbf880b5c90 2023-07-06T20:15:21.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.907+0000 7fbf8edff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf880cd490 con 0x7fbf880b6110 2023-07-06T20:15:21.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.907+0000 7fbf8edff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf880cd770 con 0x7fbf8815fca0 2023-07-06T20:15:21.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf87fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbf880b5c90 0x7fbf880d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40700/0 (socket says 172.21.15.27:40700) 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf87fff700 1 -- 172.21.15.27:0/2459768828 learned_addr learned my addr 172.21.15.27:0/2459768828 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf8cb9b700 1 --2- 172.21.15.27:0/2459768828 >> v2:172.21.15.153:3300/0 conn(0x7fbf8815fca0 0x7fbf880d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230111016 0 0) 0x7fbf880cf780 con 0x7fbf880b5c90 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf780035b0 con 0x7fbf880b5c90 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354016189 0 0) 0x7fbf880cd490 con 0x7fbf880b6110 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf880cf780 con 0x7fbf880b6110 2023-07-06T20:15:21.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf8cb9b700 1 -- 172.21.15.27:0/2459768828 >> v1:172.21.15.153:6789/0 conn(0x7fbf880b6110 legacy=0x7fbf880cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf8cb9b700 1 -- 172.21.15.27:0/2459768828 >> v1:172.21.15.27:6789/0 conn(0x7fbf880b5c90 legacy=0x7fbf880d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.908+0000 7fbf8cb9b700 1 -- 172.21.15.27:0/2459768828 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880cda50 con 0x7fbf8815fca0 2023-07-06T20:15:21.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8cb9b700 1 --2- 172.21.15.27:0/2459768828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf8815fca0 0x7fbf880d1420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009840 tx=0x7fbf7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60147a38c077e8a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:21.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c011830 con 0x7fbf8815fca0 2023-07-06T20:15:21.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c011990 con 0x7fbf8815fca0 2023-07-06T20:15:21.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf877fe700 1 -- 172.21.15.27:0/2459768828 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c011d20 con 0x7fbf8815fca0 2023-07-06T20:15:21.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8edff700 1 -- 172.21.15.27:0/2459768828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf8815fca0 msgr2=0x7fbf880d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8edff700 1 --2- 172.21.15.27:0/2459768828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf8815fca0 0x7fbf880d1420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009840 tx=0x7fbf7c009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:21.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8edff700 1 -- 172.21.15.27:0/2459768828 shutdown_connections 2023-07-06T20:15:21.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8edff700 1 --2- 172.21.15.27:0/2459768828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf8815fca0 0x7fbf880d1420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8edff700 1 -- 172.21.15.27:0/2459768828 >> 172.21.15.27:0/2459768828 conn(0x7fbf8806c150 msgr2=0x7fbf8806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:21.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.909+0000 7fbf8edff700 1 -- 172.21.15.27:0/2459768828 shutdown_connections 2023-07-06T20:15:21.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.910+0000 7fbf8edff700 1 -- 172.21.15.27:0/2459768828 wait complete. 2023-07-06T20:15:21.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.911+0000 7fbf8edff700 1 Processor -- start 2023-07-06T20:15:21.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.911+0000 7fbf8edff700 1 -- start start 2023-07-06T20:15:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.912+0000 7fbf8edff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf880b5c90 0x7fbf880cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.912+0000 7fbf8edff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf880b6110 0x7fbf881f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.912+0000 7fbf8edff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 0x7fbf881f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.912+0000 7fbf8edff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881f2db0 con 0x7fbf8815fca0 2023-07-06T20:15:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.912+0000 7fbf8edff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881f5e20 con 0x7fbf880b6110 2023-07-06T20:15:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.912+0000 7fbf8edff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881f6100 con 0x7fbf880b5c90 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 0x7fbf881f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 0x7fbf881f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45706/0 (socket says 172.21.15.27:45706) 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 -- 172.21.15.27:0/3686271892 learned_addr learned my addr 172.21.15.27:0/3686271892 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8cb9b700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf880b5c90 0x7fbf880cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf87fff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf880b6110 0x7fbf881f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 -- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf880b5c90 msgr2=0x7fbf880cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf880b5c90 0x7fbf880cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 -- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf880b6110 msgr2=0x7fbf881f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf880b6110 0x7fbf881f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 -- 172.21.15.27:0/3686271892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0094a0 con 0x7fbf8815fca0 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf87fff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf880b6110 0x7fbf881f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.913+0000 7fbf8d39c700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 0x7fbf881f53a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000cc20 tx=0x7fbf8000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.914+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80003c00 con 0x7fbf8815fca0 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.914+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80003d60 con 0x7fbf8815fca0 2023-07-06T20:15:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.914+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000ab90 con 0x7fbf8815fca0 2023-07-06T20:15:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.914+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf881f2070 con 0x7fbf8815fca0 2023-07-06T20:15:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.914+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf881f1d60 con 0x7fbf8815fca0 2023-07-06T20:15:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.915+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf881f2590 con 0x7fbf8815fca0 2023-07-06T20:15:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.916+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf8000acf0 con 0x7fbf8815fca0 2023-07-06T20:15:21.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.918+0000 7fbf857fa700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf6c06ace0 0x7fbf6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:21.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.918+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf80089500 con 0x7fbf8815fca0 2023-07-06T20:15:21.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.918+0000 7fbf8cb9b700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf6c06ace0 0x7fbf6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:21.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.919+0000 7fbf8cb9b700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf6c06ace0 0x7fbf6c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009fd0 tx=0x7fbf7c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:21.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:21.922+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf881f2590 con 0x7fbf8815fca0 2023-07-06T20:15:22.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.123+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf8806ae40 con 0x7fbf6c06ace0 2023-07-06T20:15:22.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.132+0000 7fbf857fa700 1 -- 172.21.15.27:0/3686271892 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311081 (secure 0 0 0) 0x7fbf7c011b00 con 0x7fbf6c06ace0 2023-07-06T20:15:22.137 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:22.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf6c06ace0 msgr2=0x7fbf6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf6c06ace0 0x7fbf6c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009fd0 tx=0x7fbf7c01c040 comp rx=0 tx=0).stop 2023-07-06T20:15:22.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 msgr2=0x7fbf881f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 0x7fbf881f53a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000cc20 tx=0x7fbf8000cf30 comp rx=0 tx=0).stop 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 shutdown_connections 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fbf6c06ace0 0x7fbf6c06d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf8815fca0 0x7fbf881f53a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbf880b5c90 0x7fbf880cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 --2- 172.21.15.27:0/3686271892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf880b6110 0x7fbf881f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 >> 172.21.15.27:0/3686271892 conn(0x7fbf8806c150 msgr2=0x7fbf880d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 shutdown_connections 2023-07-06T20:15:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.137+0000 7fbf8edff700 1 -- 172.21.15.27:0/3686271892 wait complete. 2023-07-06T20:15:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:15:22.157 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-07-06T20:15:20.362794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220012,"kb_used_data":4396,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754708,"statfs":{"total":773094113280,"available":772868820992,"internally_reserved":0,"allocated":4501504,"data_stored":2528895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.169091"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152170+0000","last_change":"2023-07-06T20:15:17.235418+0000","last_active":"2023-07-06T20:15:19.152170+0000","last_peered":"2023-07-06T20:15:19.152170+0000","last_clean":"2023-07-06T20:15:19.152170+0000","last_became_active":"2023-07-06T20:15:17.235083+0000","last_became_peered":"2023-07-06T20:15:17.235083+0000","last_unstale":"2023-07-06T20:15:19.152170+0000","last_undegraded":"2023-07-06T20:15:19.152170+0000","last_fullsized":"2023-07-06T20:15:19.152170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:35.832975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.151987+0000","last_change":"2023-07-06T20:15:17.229383+0000","last_active":"2023-07-06T20:15:19.151987+0000","last_peered":"2023-07-06T20:15:19.151987+0000","last_clean":"2023-07-06T20:15:19.151987+0000","last_became_active":"2023-07-06T20:15:17.229052+0000","last_became_peered":"2023-07-06T20:15:17.229052+0000","last_unstale":"2023-07-06T20:15:19.151987+0000","last_undegraded":"2023-07-06T20:15:19.151987+0000","last_fullsized":"2023-07-06T20:15:19.151987+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:27.294552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152730+0000","last_change":"2023-07-06T20:15:17.246105+0000","last_active":"2023-07-06T20:15:19.152730+0000","last_peered":"2023-07-06T20:15:19.152730+0000","last_clean":"2023-07-06T20:15:19.152730+0000","last_became_active":"2023-07-06T20:15:17.245834+0000","last_became_peered":"2023-07-06T20:15:17.245834+0000","last_unstale":"2023-07-06T20:15:19.152730+0000","last_undegraded":"2023-07-06T20:15:19.152730+0000","last_fullsized":"2023-07-06T20:15:19.152730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:57:48.651414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216966+0000","last_change":"2023-07-06T20:15:17.228851+0000","last_active":"2023-07-06T20:15:18.216966+0000","last_peered":"2023-07-06T20:15:18.216966+0000","last_clean":"2023-07-06T20:15:18.216966+0000","last_became_active":"2023-07-06T20:15:17.228586+0000","last_became_peered":"2023-07-06T20:15:17.228586+0000","last_unstale":"2023-07-06T20:15:18.216966+0000","last_undegraded":"2023-07-06T20:15:18.216966+0000","last_fullsized":"2023-07-06T20:15:18.216966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:49:20.404609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216657+0000","last_change":"2023-07-06T20:15:17.228308+0000","last_active":"2023-07-06T20:15:18.216657+0000","last_peered":"2023-07-06T20:15:18.216657+0000","last_clean":"2023-07-06T20:15:18.216657+0000","last_became_active":"2023-07-06T20:15:17.228012+0000","last_became_peered":"2023-07-06T20:15:17.228012+0000","last_unstale":"2023-07-06T20:15:18.216657+0000","last_undegraded":"2023-07-06T20:15:18.216657+0000","last_fullsized":"2023-07-06T20:15:18.216657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:04:45.731057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700882+0000","last_change":"2023-07-06T20:15:17.223324+0000","last_active":"2023-07-06T20:15:18.700882+0000","last_peered":"2023-07-06T20:15:18.700882+0000","last_clean":"2023-07-06T20:15:18.700882+0000","last_became_active":"2023-07-06T20:15:17.223065+0000","last_became_peered":"2023-07-06T20:15:17.223065+0000","last_unstale":"2023-07-06T20:15:18.700882+0000","last_undegraded":"2023-07-06T20:15:18.700882+0000","last_fullsized":"2023-07-06T20:15:18.700882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:51:28.509267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345389+0000","last_change":"2023-07-06T20:15:17.229972+0000","last_active":"2023-07-06T20:15:18.345389+0000","last_peered":"2023-07-06T20:15:18.345389+0000","last_clean":"2023-07-06T20:15:18.345389+0000","last_became_active":"2023-07-06T20:15:17.229670+0000","last_became_peered":"2023-07-06T20:15:17.229670+0000","last_unstale":"2023-07-06T20:15:18.345389+0000","last_undegraded":"2023-07-06T20:15:18.345389+0000","last_fullsized":"2023-07-06T20:15:18.345389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.909334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152386+0000","last_change":"2023-07-06T20:15:17.226201+0000","last_active":"2023-07-06T20:15:19.152386+0000","last_peered":"2023-07-06T20:15:19.152386+0000","last_clean":"2023-07-06T20:15:19.152386+0000","last_became_active":"2023-07-06T20:15:17.225906+0000","last_became_peered":"2023-07-06T20:15:17.225906+0000","last_unstale":"2023-07-06T20:15:19.152386+0000","last_undegraded":"2023-07-06T20:15:19.152386+0000","last_fullsized":"2023-07-06T20:15:19.152386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:59.691407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152873+0000","last_change":"2023-07-06T20:15:17.246977+0000","last_active":"2023-07-06T20:15:19.152873+0000","last_peered":"2023-07-06T20:15:19.152873+0000","last_clean":"2023-07-06T20:15:19.152873+0000","last_became_active":"2023-07-06T20:15:17.246690+0000","last_became_peered":"2023-07-06T20:15:17.246690+0000","last_unstale":"2023-07-06T20:15:19.152873+0000","last_undegraded":"2023-07-06T20:15:19.152873+0000","last_fullsized":"2023-07-06T20:15:19.152873+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:40:08.053320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.346007+0000","last_change":"2023-07-06T20:15:17.237522+0000","last_active":"2023-07-06T20:15:18.346007+0000","last_peered":"2023-07-06T20:15:18.346007+0000","last_clean":"2023-07-06T20:15:18.346007+0000","last_became_active":"2023-07-06T20:15:17.237289+0000","last_became_peered":"2023-07-06T20:15:17.237289+0000","last_unstale":"2023-07-06T20:15:18.346007+0000","last_undegraded":"2023-07-06T20:15:18.346007+0000","last_fullsized":"2023-07-06T20:15:18.346007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:30:07.755446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345735+0000","last_change":"2023-07-06T20:15:17.232247+0000","last_active":"2023-07-06T20:15:18.345735+0000","last_peered":"2023-07-06T20:15:18.345735+0000","last_clean":"2023-07-06T20:15:18.345735+0000","last_became_active":"2023-07-06T20:15:17.231992+0000","last_became_peered":"2023-07-06T20:15:17.231992+0000","last_unstale":"2023-07-06T20:15:18.345735+0000","last_undegraded":"2023-07-06T20:15:18.345735+0000","last_fullsized":"2023-07-06T20:15:18.345735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:02:25.230516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152742+0000","last_change":"2023-07-06T20:15:17.236809+0000","last_active":"2023-07-06T20:15:19.152742+0000","last_peered":"2023-07-06T20:15:19.152742+0000","last_clean":"2023-07-06T20:15:19.152742+0000","last_became_active":"2023-07-06T20:15:17.235497+0000","last_became_peered":"2023-07-06T20:15:17.235497+0000","last_unstale":"2023-07-06T20:15:19.152742+0000","last_undegraded":"2023-07-06T20:15:19.152742+0000","last_fullsized":"2023-07-06T20:15:19.152742+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:49:34.644264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345387+0000","last_change":"2023-07-06T20:15:17.240702+0000","last_active":"2023-07-06T20:15:18.345387+0000","last_peered":"2023-07-06T20:15:18.345387+0000","last_clean":"2023-07-06T20:15:18.345387+0000","last_became_active":"2023-07-06T20:15:17.240399+0000","last_became_peered":"2023-07-06T20:15:17.240399+0000","last_unstale":"2023-07-06T20:15:18.345387+0000","last_undegraded":"2023-07-06T20:15:18.345387+0000","last_fullsized":"2023-07-06T20:15:18.345387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:37.253145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453717+0000","last_change":"2023-07-06T20:15:17.242077+0000","last_active":"2023-07-06T20:15:18.453717+0000","last_peered":"2023-07-06T20:15:18.453717+0000","last_clean":"2023-07-06T20:15:18.453717+0000","last_became_active":"2023-07-06T20:15:17.241776+0000","last_became_peered":"2023-07-06T20:15:17.241776+0000","last_unstale":"2023-07-06T20:15:18.453717+0000","last_undegraded":"2023-07-06T20:15:18.453717+0000","last_fullsized":"2023-07-06T20:15:18.453717+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:34:54.294041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345154+0000","last_change":"2023-07-06T20:15:17.232316+0000","last_active":"2023-07-06T20:15:18.345154+0000","last_peered":"2023-07-06T20:15:18.345154+0000","last_clean":"2023-07-06T20:15:18.345154+0000","last_became_active":"2023-07-06T20:15:17.232069+0000","last_became_peered":"2023-07-06T20:15:17.232069+0000","last_unstale":"2023-07-06T20:15:18.345154+0000","last_undegraded":"2023-07-06T20:15:18.345154+0000","last_fullsized":"2023-07-06T20:15:18.345154+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:35.832975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322254+0000","last_change":"2023-07-06T20:15:17.246820+0000","last_active":"2023-07-06T20:15:18.322254+0000","last_peered":"2023-07-06T20:15:18.322254+0000","last_clean":"2023-07-06T20:15:18.322254+0000","last_became_active":"2023-07-06T20:15:17.246540+0000","last_became_peered":"2023-07-06T20:15:17.246540+0000","last_unstale":"2023-07-06T20:15:18.322254+0000","last_undegraded":"2023-07-06T20:15:18.322254+0000","last_fullsized":"2023-07-06T20:15:18.322254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:02:45.009876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322714+0000","last_change":"2023-07-06T20:15:17.246391+0000","last_active":"2023-07-06T20:15:18.322714+0000","last_peered":"2023-07-06T20:15:18.322714+0000","last_clean":"2023-07-06T20:15:18.322714+0000","last_became_active":"2023-07-06T20:15:17.246107+0000","last_became_peered":"2023-07-06T20:15:17.246107+0000","last_unstale":"2023-07-06T20:15:18.322714+0000","last_undegraded":"2023-07-06T20:15:18.322714+0000","last_fullsized":"2023-07-06T20:15:18.322714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:54:42.425621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218412+0000","last_change":"2023-07-06T20:15:17.243903+0000","last_active":"2023-07-06T20:15:18.218412+0000","last_peered":"2023-07-06T20:15:18.218412+0000","last_clean":"2023-07-06T20:15:18.218412+0000","last_became_active":"2023-07-06T20:15:17.242739+0000","last_became_peered":"2023-07-06T20:15:17.242739+0000","last_unstale":"2023-07-06T20:15:18.218412+0000","last_undegraded":"2023-07-06T20:15:18.218412+0000","last_fullsized":"2023-07-06T20:15:18.218412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:40.235154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216880+0000","last_change":"2023-07-06T20:15:17.223730+0000","last_active":"2023-07-06T20:15:18.216880+0000","last_peered":"2023-07-06T20:15:18.216880+0000","last_clean":"2023-07-06T20:15:18.216880+0000","last_became_active":"2023-07-06T20:15:17.223377+0000","last_became_peered":"2023-07-06T20:15:17.223377+0000","last_unstale":"2023-07-06T20:15:18.216880+0000","last_undegraded":"2023-07-06T20:15:18.216880+0000","last_fullsized":"2023-07-06T20:15:18.216880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:47:53.614657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218221+0000","last_change":"2023-07-06T20:15:17.242985+0000","last_active":"2023-07-06T20:15:18.218221+0000","last_peered":"2023-07-06T20:15:18.218221+0000","last_clean":"2023-07-06T20:15:18.218221+0000","last_became_active":"2023-07-06T20:15:17.242599+0000","last_became_peered":"2023-07-06T20:15:17.242599+0000","last_unstale":"2023-07-06T20:15:18.218221+0000","last_undegraded":"2023-07-06T20:15:18.218221+0000","last_fullsized":"2023-07-06T20:15:18.218221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:25.185264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345505+0000","last_change":"2023-07-06T20:15:17.227709+0000","last_active":"2023-07-06T20:15:18.345505+0000","last_peered":"2023-07-06T20:15:18.345505+0000","last_clean":"2023-07-06T20:15:18.345505+0000","last_became_active":"2023-07-06T20:15:17.227450+0000","last_became_peered":"2023-07-06T20:15:17.227450+0000","last_unstale":"2023-07-06T20:15:18.345505+0000","last_undegraded":"2023-07-06T20:15:18.345505+0000","last_fullsized":"2023-07-06T20:15:18.345505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:02:45.009876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453278+0000","last_change":"2023-07-06T20:15:17.235436+0000","last_active":"2023-07-06T20:15:18.453278+0000","last_peered":"2023-07-06T20:15:18.453278+0000","last_clean":"2023-07-06T20:15:18.453278+0000","last_became_active":"2023-07-06T20:15:17.235038+0000","last_became_peered":"2023-07-06T20:15:17.235038+0000","last_unstale":"2023-07-06T20:15:18.453278+0000","last_undegraded":"2023-07-06T20:15:18.453278+0000","last_fullsized":"2023-07-06T20:15:18.453278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:03:44.134546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322657+0000","last_change":"2023-07-06T20:15:17.238641+0000","last_active":"2023-07-06T20:15:18.322657+0000","last_peered":"2023-07-06T20:15:18.322657+0000","last_clean":"2023-07-06T20:15:18.322657+0000","last_became_active":"2023-07-06T20:15:17.238349+0000","last_became_peered":"2023-07-06T20:15:17.238349+0000","last_unstale":"2023-07-06T20:15:18.322657+0000","last_undegraded":"2023-07-06T20:15:18.322657+0000","last_fullsized":"2023-07-06T20:15:18.322657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:07.779943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216736+0000","last_change":"2023-07-06T20:15:17.239695+0000","last_active":"2023-07-06T20:15:18.216736+0000","last_peered":"2023-07-06T20:15:18.216736+0000","last_clean":"2023-07-06T20:15:18.216736+0000","last_became_active":"2023-07-06T20:15:17.238712+0000","last_became_peered":"2023-07-06T20:15:17.238712+0000","last_unstale":"2023-07-06T20:15:18.216736+0000","last_undegraded":"2023-07-06T20:15:18.216736+0000","last_fullsized":"2023-07-06T20:15:18.216736+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:25.185264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218432+0000","last_change":"2023-07-06T20:15:17.238232+0000","last_active":"2023-07-06T20:15:18.218432+0000","last_peered":"2023-07-06T20:15:18.218432+0000","last_clean":"2023-07-06T20:15:18.218432+0000","last_became_active":"2023-07-06T20:15:17.237964+0000","last_became_peered":"2023-07-06T20:15:17.237964+0000","last_unstale":"2023-07-06T20:15:18.218432+0000","last_undegraded":"2023-07-06T20:15:18.218432+0000","last_fullsized":"2023-07-06T20:15:18.218432+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:00:22.922695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701142+0000","last_change":"2023-07-06T20:15:17.227524+0000","last_active":"2023-07-06T20:15:18.701142+0000","last_peered":"2023-07-06T20:15:18.701142+0000","last_clean":"2023-07-06T20:15:18.701142+0000","last_became_active":"2023-07-06T20:15:17.227258+0000","last_became_peered":"2023-07-06T20:15:17.227258+0000","last_unstale":"2023-07-06T20:15:18.701142+0000","last_undegraded":"2023-07-06T20:15:18.701142+0000","last_fullsized":"2023-07-06T20:15:18.701142+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:35.832975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218550+0000","last_change":"2023-07-06T20:15:17.237130+0000","last_active":"2023-07-06T20:15:18.218550+0000","last_peered":"2023-07-06T20:15:18.218550+0000","last_clean":"2023-07-06T20:15:18.218550+0000","last_became_active":"2023-07-06T20:15:17.236841+0000","last_became_peered":"2023-07-06T20:15:17.236841+0000","last_unstale":"2023-07-06T20:15:18.218550+0000","last_undegraded":"2023-07-06T20:15:18.218550+0000","last_fullsized":"2023-07-06T20:15:18.218550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:19:05.597036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345603+0000","last_change":"2023-07-06T20:15:17.224030+0000","last_active":"2023-07-06T20:15:18.345603+0000","last_peered":"2023-07-06T20:15:18.345603+0000","last_clean":"2023-07-06T20:15:18.345603+0000","last_became_active":"2023-07-06T20:15:17.223733+0000","last_became_peered":"2023-07-06T20:15:17.223733+0000","last_unstale":"2023-07-06T20:15:18.345603+0000","last_undegraded":"2023-07-06T20:15:18.345603+0000","last_fullsized":"2023-07-06T20:15:18.345603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:37:01.758729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218391+0000","last_change":"2023-07-06T20:15:17.240811+0000","last_active":"2023-07-06T20:15:18.218391+0000","last_peered":"2023-07-06T20:15:18.218391+0000","last_clean":"2023-07-06T20:15:18.218391+0000","last_became_active":"2023-07-06T20:15:17.240289+0000","last_became_peered":"2023-07-06T20:15:17.240289+0000","last_unstale":"2023-07-06T20:15:18.218391+0000","last_undegraded":"2023-07-06T20:15:18.218391+0000","last_fullsized":"2023-07-06T20:15:18.218391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:30:03.586651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218260+0000","last_change":"2023-07-06T20:15:17.243674+0000","last_active":"2023-07-06T20:15:18.218260+0000","last_peered":"2023-07-06T20:15:18.218260+0000","last_clean":"2023-07-06T20:15:18.218260+0000","last_became_active":"2023-07-06T20:15:17.243391+0000","last_became_peered":"2023-07-06T20:15:17.243391+0000","last_unstale":"2023-07-06T20:15:18.218260+0000","last_undegraded":"2023-07-06T20:15:18.218260+0000","last_fullsized":"2023-07-06T20:15:18.218260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:47:53.614657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322342+0000","last_change":"2023-07-06T20:15:17.236869+0000","last_active":"2023-07-06T20:15:18.322342+0000","last_peered":"2023-07-06T20:15:18.322342+0000","last_clean":"2023-07-06T20:15:18.322342+0000","last_became_active":"2023-07-06T20:15:17.236587+0000","last_became_peered":"2023-07-06T20:15:17.236587+0000","last_unstale":"2023-07-06T20:15:18.322342+0000","last_undegraded":"2023-07-06T20:15:18.322342+0000","last_fullsized":"2023-07-06T20:15:18.322342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:37:01.758729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700819+0000","last_change":"2023-07-06T20:15:17.222615+0000","last_active":"2023-07-06T20:15:18.700819+0000","last_peered":"2023-07-06T20:15:18.700819+0000","last_clean":"2023-07-06T20:15:18.700819+0000","last_became_active":"2023-07-06T20:15:17.222337+0000","last_became_peered":"2023-07-06T20:15:17.222337+0000","last_unstale":"2023-07-06T20:15:18.700819+0000","last_undegraded":"2023-07-06T20:15:18.700819+0000","last_fullsized":"2023-07-06T20:15:18.700819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:59.691407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322714+0000","last_change":"2023-07-06T20:15:17.245128+0000","last_active":"2023-07-06T20:15:18.322714+0000","last_peered":"2023-07-06T20:15:18.322714+0000","last_clean":"2023-07-06T20:15:18.322714+0000","last_became_active":"2023-07-06T20:15:17.244812+0000","last_became_peered":"2023-07-06T20:15:17.244812+0000","last_unstale":"2023-07-06T20:15:18.322714+0000","last_undegraded":"2023-07-06T20:15:18.322714+0000","last_fullsized":"2023-07-06T20:15:18.322714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:14:41.364559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218429+0000","last_change":"2023-07-06T20:15:16.226731+0000","last_active":"2023-07-06T20:15:18.218429+0000","last_peered":"2023-07-06T20:15:18.218429+0000","last_clean":"2023-07-06T20:15:18.218429+0000","last_became_active":"2023-07-06T20:15:16.226436+0000","last_became_peered":"2023-07-06T20:15:16.226436+0000","last_unstale":"2023-07-06T20:15:18.218429+0000","last_undegraded":"2023-07-06T20:15:18.218429+0000","last_fullsized":"2023-07-06T20:15:18.218429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:30:12.897990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701226+0000","last_change":"2023-07-06T20:15:17.226548+0000","last_active":"2023-07-06T20:15:18.701226+0000","last_peered":"2023-07-06T20:15:18.701226+0000","last_clean":"2023-07-06T20:15:18.701226+0000","last_became_active":"2023-07-06T20:15:17.226270+0000","last_became_peered":"2023-07-06T20:15:17.226270+0000","last_unstale":"2023-07-06T20:15:18.701226+0000","last_undegraded":"2023-07-06T20:15:18.701226+0000","last_fullsized":"2023-07-06T20:15:18.701226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.909334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152364+0000","last_change":"2023-07-06T20:15:16.222300+0000","last_active":"2023-07-06T20:15:19.152364+0000","last_peered":"2023-07-06T20:15:19.152364+0000","last_clean":"2023-07-06T20:15:19.152364+0000","last_became_active":"2023-07-06T20:15:16.222038+0000","last_became_peered":"2023-07-06T20:15:16.222038+0000","last_unstale":"2023-07-06T20:15:19.152364+0000","last_undegraded":"2023-07-06T20:15:19.152364+0000","last_fullsized":"2023-07-06T20:15:19.152364+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:19:02.243527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453675+0000","last_change":"2023-07-06T20:15:17.240535+0000","last_active":"2023-07-06T20:15:18.453675+0000","last_peered":"2023-07-06T20:15:18.453675+0000","last_clean":"2023-07-06T20:15:18.453675+0000","last_became_active":"2023-07-06T20:15:17.240252+0000","last_became_peered":"2023-07-06T20:15:17.240252+0000","last_unstale":"2023-07-06T20:15:18.453675+0000","last_undegraded":"2023-07-06T20:15:18.453675+0000","last_fullsized":"2023-07-06T20:15:18.453675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:19.450885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701253+0000","last_change":"2023-07-06T20:15:16.219402+0000","last_active":"2023-07-06T20:15:18.701253+0000","last_peered":"2023-07-06T20:15:18.701253+0000","last_clean":"2023-07-06T20:15:18.701253+0000","last_became_active":"2023-07-06T20:15:16.219073+0000","last_became_peered":"2023-07-06T20:15:16.219073+0000","last_unstale":"2023-07-06T20:15:18.701253+0000","last_undegraded":"2023-07-06T20:15:18.701253+0000","last_fullsized":"2023-07-06T20:15:18.701253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:16.795193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152186+0000","last_change":"2023-07-06T20:15:17.238324+0000","last_active":"2023-07-06T20:15:19.152186+0000","last_peered":"2023-07-06T20:15:19.152186+0000","last_clean":"2023-07-06T20:15:19.152186+0000","last_became_active":"2023-07-06T20:15:17.238109+0000","last_became_peered":"2023-07-06T20:15:17.238109+0000","last_unstale":"2023-07-06T20:15:19.152186+0000","last_undegraded":"2023-07-06T20:15:19.152186+0000","last_fullsized":"2023-07-06T20:15:19.152186+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:30.023080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453479+0000","last_change":"2023-07-06T20:15:16.217117+0000","last_active":"2023-07-06T20:15:18.453479+0000","last_peered":"2023-07-06T20:15:18.453479+0000","last_clean":"2023-07-06T20:15:18.453479+0000","last_became_active":"2023-07-06T20:15:16.214703+0000","last_became_peered":"2023-07-06T20:15:16.214703+0000","last_unstale":"2023-07-06T20:15:18.453479+0000","last_undegraded":"2023-07-06T20:15:18.453479+0000","last_fullsized":"2023-07-06T20:15:18.453479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:28:59.869660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152238+0000","last_change":"2023-07-06T20:15:17.241995+0000","last_active":"2023-07-06T20:15:19.152238+0000","last_peered":"2023-07-06T20:15:19.152238+0000","last_clean":"2023-07-06T20:15:19.152238+0000","last_became_active":"2023-07-06T20:15:17.241705+0000","last_became_peered":"2023-07-06T20:15:17.241705+0000","last_unstale":"2023-07-06T20:15:19.152238+0000","last_undegraded":"2023-07-06T20:15:19.152238+0000","last_fullsized":"2023-07-06T20:15:19.152238+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:27:58.742569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345751+0000","last_change":"2023-07-06T20:15:16.220418+0000","last_active":"2023-07-06T20:15:18.345751+0000","last_peered":"2023-07-06T20:15:18.345751+0000","last_clean":"2023-07-06T20:15:18.345751+0000","last_became_active":"2023-07-06T20:15:16.220049+0000","last_became_peered":"2023-07-06T20:15:16.220049+0000","last_unstale":"2023-07-06T20:15:18.345751+0000","last_undegraded":"2023-07-06T20:15:18.345751+0000","last_fullsized":"2023-07-06T20:15:18.345751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:04.894008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345619+0000","last_change":"2023-07-06T20:15:17.231816+0000","last_active":"2023-07-06T20:15:18.345619+0000","last_peered":"2023-07-06T20:15:18.345619+0000","last_clean":"2023-07-06T20:15:18.345619+0000","last_became_active":"2023-07-06T20:15:17.231525+0000","last_became_peered":"2023-07-06T20:15:17.231525+0000","last_unstale":"2023-07-06T20:15:18.345619+0000","last_undegraded":"2023-07-06T20:15:18.345619+0000","last_fullsized":"2023-07-06T20:15:18.345619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:17.802296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322342+0000","last_change":"2023-07-06T20:15:16.226735+0000","last_active":"2023-07-06T20:15:18.322342+0000","last_peered":"2023-07-06T20:15:18.322342+0000","last_clean":"2023-07-06T20:15:18.322342+0000","last_became_active":"2023-07-06T20:15:16.226454+0000","last_became_peered":"2023-07-06T20:15:16.226454+0000","last_unstale":"2023-07-06T20:15:18.322342+0000","last_undegraded":"2023-07-06T20:15:18.322342+0000","last_fullsized":"2023-07-06T20:15:18.322342+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:42.703860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453277+0000","last_change":"2023-07-06T20:15:17.235861+0000","last_active":"2023-07-06T20:15:18.453277+0000","last_peered":"2023-07-06T20:15:18.453277+0000","last_clean":"2023-07-06T20:15:18.453277+0000","last_became_active":"2023-07-06T20:15:17.235525+0000","last_became_peered":"2023-07-06T20:15:17.235525+0000","last_unstale":"2023-07-06T20:15:18.453277+0000","last_undegraded":"2023-07-06T20:15:18.453277+0000","last_fullsized":"2023-07-06T20:15:18.453277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:00.880380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700939+0000","last_change":"2023-07-06T20:15:16.221347+0000","last_active":"2023-07-06T20:15:18.700939+0000","last_peered":"2023-07-06T20:15:18.700939+0000","last_clean":"2023-07-06T20:15:18.700939+0000","last_became_active":"2023-07-06T20:15:16.221064+0000","last_became_peered":"2023-07-06T20:15:16.221064+0000","last_unstale":"2023-07-06T20:15:18.700939+0000","last_undegraded":"2023-07-06T20:15:18.700939+0000","last_fullsized":"2023-07-06T20:15:18.700939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:26.287448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152349+0000","last_change":"2023-07-06T20:15:17.243169+0000","last_active":"2023-07-06T20:15:19.152349+0000","last_peered":"2023-07-06T20:15:19.152349+0000","last_clean":"2023-07-06T20:15:19.152349+0000","last_became_active":"2023-07-06T20:15:17.242838+0000","last_became_peered":"2023-07-06T20:15:17.242838+0000","last_unstale":"2023-07-06T20:15:19.152349+0000","last_undegraded":"2023-07-06T20:15:19.152349+0000","last_fullsized":"2023-07-06T20:15:19.152349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:17.802296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453240+0000","last_change":"2023-07-06T20:15:16.221133+0000","last_active":"2023-07-06T20:15:18.453240+0000","last_peered":"2023-07-06T20:15:18.453240+0000","last_clean":"2023-07-06T20:15:18.453240+0000","last_became_active":"2023-07-06T20:15:16.220855+0000","last_became_peered":"2023-07-06T20:15:16.220855+0000","last_unstale":"2023-07-06T20:15:18.453240+0000","last_undegraded":"2023-07-06T20:15:18.453240+0000","last_fullsized":"2023-07-06T20:15:18.453240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:18:27.414965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.217000+0000","last_change":"2023-07-06T20:15:17.245308+0000","last_active":"2023-07-06T20:15:18.217000+0000","last_peered":"2023-07-06T20:15:18.217000+0000","last_clean":"2023-07-06T20:15:18.217000+0000","last_became_active":"2023-07-06T20:15:17.245040+0000","last_became_peered":"2023-07-06T20:15:17.245040+0000","last_unstale":"2023-07-06T20:15:18.217000+0000","last_undegraded":"2023-07-06T20:15:18.217000+0000","last_fullsized":"2023-07-06T20:15:18.217000+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:40.235154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152265+0000","last_change":"2023-07-06T20:15:16.210293+0000","last_active":"2023-07-06T20:15:19.152265+0000","last_peered":"2023-07-06T20:15:19.152265+0000","last_clean":"2023-07-06T20:15:19.152265+0000","last_became_active":"2023-07-06T20:15:16.209994+0000","last_became_peered":"2023-07-06T20:15:16.209994+0000","last_unstale":"2023-07-06T20:15:19.152265+0000","last_undegraded":"2023-07-06T20:15:19.152265+0000","last_fullsized":"2023-07-06T20:15:19.152265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:12.902230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322584+0000","last_change":"2023-07-06T20:15:17.241717+0000","last_active":"2023-07-06T20:15:18.322584+0000","last_peered":"2023-07-06T20:15:18.322584+0000","last_clean":"2023-07-06T20:15:18.322584+0000","last_became_active":"2023-07-06T20:15:17.241455+0000","last_became_peered":"2023-07-06T20:15:17.241455+0000","last_unstale":"2023-07-06T20:15:18.322584+0000","last_undegraded":"2023-07-06T20:15:18.322584+0000","last_fullsized":"2023-07-06T20:15:18.322584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:05.901112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345928+0000","last_change":"2023-07-06T20:15:16.216548+0000","last_active":"2023-07-06T20:15:18.345928+0000","last_peered":"2023-07-06T20:15:18.345928+0000","last_clean":"2023-07-06T20:15:18.345928+0000","last_became_active":"2023-07-06T20:15:16.215145+0000","last_became_peered":"2023-07-06T20:15:16.215145+0000","last_unstale":"2023-07-06T20:15:18.345928+0000","last_undegraded":"2023-07-06T20:15:18.345928+0000","last_fullsized":"2023-07-06T20:15:18.345928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:06.772839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152211+0000","last_change":"2023-07-06T20:15:17.237487+0000","last_active":"2023-07-06T20:15:19.152211+0000","last_peered":"2023-07-06T20:15:19.152211+0000","last_clean":"2023-07-06T20:15:19.152211+0000","last_became_active":"2023-07-06T20:15:17.237204+0000","last_became_peered":"2023-07-06T20:15:17.237204+0000","last_unstale":"2023-07-06T20:15:19.152211+0000","last_undegraded":"2023-07-06T20:15:19.152211+0000","last_fullsized":"2023-07-06T20:15:19.152211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:13.846737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345831+0000","last_change":"2023-07-06T20:15:16.218752+0000","last_active":"2023-07-06T20:15:18.345831+0000","last_peered":"2023-07-06T20:15:18.345831+0000","last_clean":"2023-07-06T20:15:18.345831+0000","last_became_active":"2023-07-06T20:15:16.218262+0000","last_became_peered":"2023-07-06T20:15:16.218262+0000","last_unstale":"2023-07-06T20:15:18.345831+0000","last_undegraded":"2023-07-06T20:15:18.345831+0000","last_fullsized":"2023-07-06T20:15:18.345831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:14:40.357455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453842+0000","last_change":"2023-07-06T20:15:17.235525+0000","last_active":"2023-07-06T20:15:18.453842+0000","last_peered":"2023-07-06T20:15:18.453842+0000","last_clean":"2023-07-06T20:15:18.453842+0000","last_became_active":"2023-07-06T20:15:17.235191+0000","last_became_peered":"2023-07-06T20:15:17.235191+0000","last_unstale":"2023-07-06T20:15:18.453842+0000","last_undegraded":"2023-07-06T20:15:18.453842+0000","last_fullsized":"2023-07-06T20:15:18.453842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:40:36.235206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152580+0000","last_change":"2023-07-06T20:15:16.220025+0000","last_active":"2023-07-06T20:15:19.152580+0000","last_peered":"2023-07-06T20:15:19.152580+0000","last_clean":"2023-07-06T20:15:19.152580+0000","last_became_active":"2023-07-06T20:15:16.219733+0000","last_became_peered":"2023-07-06T20:15:16.219733+0000","last_unstale":"2023-07-06T20:15:19.152580+0000","last_undegraded":"2023-07-06T20:15:19.152580+0000","last_fullsized":"2023-07-06T20:15:19.152580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:34:48.403559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152469+0000","last_change":"2023-07-06T20:15:17.246117+0000","last_active":"2023-07-06T20:15:19.152469+0000","last_peered":"2023-07-06T20:15:19.152469+0000","last_clean":"2023-07-06T20:15:19.152469+0000","last_became_active":"2023-07-06T20:15:17.245802+0000","last_became_peered":"2023-07-06T20:15:17.245802+0000","last_unstale":"2023-07-06T20:15:19.152469+0000","last_undegraded":"2023-07-06T20:15:19.152469+0000","last_fullsized":"2023-07-06T20:15:19.152469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:48:52.572201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152475+0000","last_change":"2023-07-06T20:15:16.217580+0000","last_active":"2023-07-06T20:15:19.152475+0000","last_peered":"2023-07-06T20:15:19.152475+0000","last_clean":"2023-07-06T20:15:19.152475+0000","last_became_active":"2023-07-06T20:15:16.217319+0000","last_became_peered":"2023-07-06T20:15:16.217319+0000","last_unstale":"2023-07-06T20:15:19.152475+0000","last_undegraded":"2023-07-06T20:15:19.152475+0000","last_fullsized":"2023-07-06T20:15:19.152475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:02:44.002773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152031+0000","last_change":"2023-07-06T20:15:17.233452+0000","last_active":"2023-07-06T20:15:19.152031+0000","last_peered":"2023-07-06T20:15:19.152031+0000","last_clean":"2023-07-06T20:15:19.152031+0000","last_became_active":"2023-07-06T20:15:17.233213+0000","last_became_peered":"2023-07-06T20:15:17.233213+0000","last_unstale":"2023-07-06T20:15:19.152031+0000","last_undegraded":"2023-07-06T20:15:19.152031+0000","last_fullsized":"2023-07-06T20:15:19.152031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:13:42.344623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700993+0000","last_change":"2023-07-06T20:15:16.221151+0000","last_active":"2023-07-06T20:15:18.700993+0000","last_peered":"2023-07-06T20:15:18.700993+0000","last_clean":"2023-07-06T20:15:18.700993+0000","last_became_active":"2023-07-06T20:15:16.220895+0000","last_became_peered":"2023-07-06T20:15:16.220895+0000","last_unstale":"2023-07-06T20:15:18.700993+0000","last_undegraded":"2023-07-06T20:15:18.700993+0000","last_fullsized":"2023-07-06T20:15:18.700993+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:48.762899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322646+0000","last_change":"2023-07-06T20:15:17.238985+0000","last_active":"2023-07-06T20:15:18.322646+0000","last_peered":"2023-07-06T20:15:18.322646+0000","last_clean":"2023-07-06T20:15:18.322646+0000","last_became_active":"2023-07-06T20:15:17.238701+0000","last_became_peered":"2023-07-06T20:15:17.238701+0000","last_unstale":"2023-07-06T20:15:18.322646+0000","last_undegraded":"2023-07-06T20:15:18.322646+0000","last_fullsized":"2023-07-06T20:15:18.322646+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:10.099381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.217961+0000","last_change":"2023-07-06T20:15:16.226686+0000","last_active":"2023-07-06T20:15:18.217961+0000","last_peered":"2023-07-06T20:15:18.217961+0000","last_clean":"2023-07-06T20:15:18.217961+0000","last_became_active":"2023-07-06T20:15:16.226395+0000","last_became_peered":"2023-07-06T20:15:16.226395+0000","last_unstale":"2023-07-06T20:15:18.217961+0000","last_undegraded":"2023-07-06T20:15:18.217961+0000","last_fullsized":"2023-07-06T20:15:18.217961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:59.015168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152012+0000","last_change":"2023-07-06T20:15:17.237561+0000","last_active":"2023-07-06T20:15:19.152012+0000","last_peered":"2023-07-06T20:15:19.152012+0000","last_clean":"2023-07-06T20:15:19.152012+0000","last_became_active":"2023-07-06T20:15:17.237249+0000","last_became_peered":"2023-07-06T20:15:17.237249+0000","last_unstale":"2023-07-06T20:15:19.152012+0000","last_undegraded":"2023-07-06T20:15:19.152012+0000","last_fullsized":"2023-07-06T20:15:19.152012+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:49.770003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322439+0000","last_change":"2023-07-06T20:15:16.222284+0000","last_active":"2023-07-06T20:15:18.322439+0000","last_peered":"2023-07-06T20:15:18.322439+0000","last_clean":"2023-07-06T20:15:18.322439+0000","last_became_active":"2023-07-06T20:15:16.222004+0000","last_became_peered":"2023-07-06T20:15:16.222004+0000","last_unstale":"2023-07-06T20:15:18.322439+0000","last_undegraded":"2023-07-06T20:15:18.322439+0000","last_fullsized":"2023-07-06T20:15:18.322439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:02:24.223412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152527+0000","last_change":"2023-07-06T20:15:17.240664+0000","last_active":"2023-07-06T20:15:19.152527+0000","last_peered":"2023-07-06T20:15:19.152527+0000","last_clean":"2023-07-06T20:15:19.152527+0000","last_became_active":"2023-07-06T20:15:17.240366+0000","last_became_peered":"2023-07-06T20:15:17.240366+0000","last_unstale":"2023-07-06T20:15:19.152527+0000","last_undegraded":"2023-07-06T20:15:19.152527+0000","last_fullsized":"2023-07-06T20:15:19.152527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:10.099381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.346073+0000","last_change":"2023-07-06T20:15:16.224694+0000","last_active":"2023-07-06T20:15:18.346073+0000","last_peered":"2023-07-06T20:15:18.346073+0000","last_clean":"2023-07-06T20:15:18.346073+0000","last_became_active":"2023-07-06T20:15:16.224368+0000","last_became_peered":"2023-07-06T20:15:16.224368+0000","last_unstale":"2023-07-06T20:15:18.346073+0000","last_undegraded":"2023-07-06T20:15:18.346073+0000","last_fullsized":"2023-07-06T20:15:18.346073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:54:41.418518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345667+0000","last_change":"2023-07-06T20:15:17.231710+0000","last_active":"2023-07-06T20:15:18.345667+0000","last_peered":"2023-07-06T20:15:18.345667+0000","last_clean":"2023-07-06T20:15:18.345667+0000","last_became_active":"2023-07-06T20:15:17.229055+0000","last_became_peered":"2023-07-06T20:15:17.229055+0000","last_unstale":"2023-07-06T20:15:18.345667+0000","last_undegraded":"2023-07-06T20:15:18.345667+0000","last_fullsized":"2023-07-06T20:15:18.345667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:14:43.710964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453814+0000","last_change":"2023-07-06T20:15:16.219291+0000","last_active":"2023-07-06T20:15:18.453814+0000","last_peered":"2023-07-06T20:15:18.453814+0000","last_clean":"2023-07-06T20:15:18.453814+0000","last_became_active":"2023-07-06T20:15:16.218872+0000","last_became_peered":"2023-07-06T20:15:16.218872+0000","last_unstale":"2023-07-06T20:15:18.453814+0000","last_undegraded":"2023-07-06T20:15:18.453814+0000","last_fullsized":"2023-07-06T20:15:18.453814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:40.992338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453699+0000","last_change":"2023-07-06T20:15:17.237273+0000","last_active":"2023-07-06T20:15:18.453699+0000","last_peered":"2023-07-06T20:15:18.453699+0000","last_clean":"2023-07-06T20:15:18.453699+0000","last_became_active":"2023-07-06T20:15:17.236996+0000","last_became_peered":"2023-07-06T20:15:17.236996+0000","last_unstale":"2023-07-06T20:15:18.453699+0000","last_undegraded":"2023-07-06T20:15:18.453699+0000","last_fullsized":"2023-07-06T20:15:18.453699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:53:52.736942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216675+0000","last_change":"2023-07-06T20:15:16.219836+0000","last_active":"2023-07-06T20:15:18.216675+0000","last_peered":"2023-07-06T20:15:18.216675+0000","last_clean":"2023-07-06T20:15:18.216675+0000","last_became_active":"2023-07-06T20:15:16.219564+0000","last_became_peered":"2023-07-06T20:15:16.219564+0000","last_unstale":"2023-07-06T20:15:18.216675+0000","last_undegraded":"2023-07-06T20:15:18.216675+0000","last_fullsized":"2023-07-06T20:15:18.216675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:44:28.034228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216997+0000","last_change":"2023-07-06T20:15:17.225239+0000","last_active":"2023-07-06T20:15:18.216997+0000","last_peered":"2023-07-06T20:15:18.216997+0000","last_clean":"2023-07-06T20:15:18.216997+0000","last_became_active":"2023-07-06T20:15:17.224941+0000","last_became_peered":"2023-07-06T20:15:17.224941+0000","last_unstale":"2023-07-06T20:15:18.216997+0000","last_undegraded":"2023-07-06T20:15:18.216997+0000","last_fullsized":"2023-07-06T20:15:18.216997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:30:03.586651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701179+0000","last_change":"2023-07-06T20:15:16.219300+0000","last_active":"2023-07-06T20:15:18.701179+0000","last_peered":"2023-07-06T20:15:18.701179+0000","last_clean":"2023-07-06T20:15:18.701179+0000","last_became_active":"2023-07-06T20:15:16.218906+0000","last_became_peered":"2023-07-06T20:15:16.218906+0000","last_unstale":"2023-07-06T20:15:18.701179+0000","last_undegraded":"2023-07-06T20:15:18.701179+0000","last_fullsized":"2023-07-06T20:15:18.701179+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:36.246041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322441+0000","last_change":"2023-07-06T20:15:17.236062+0000","last_active":"2023-07-06T20:15:18.322441+0000","last_peered":"2023-07-06T20:15:18.322441+0000","last_clean":"2023-07-06T20:15:18.322441+0000","last_became_active":"2023-07-06T20:15:17.235767+0000","last_became_peered":"2023-07-06T20:15:17.235767+0000","last_unstale":"2023-07-06T20:15:18.322441+0000","last_undegraded":"2023-07-06T20:15:18.322441+0000","last_fullsized":"2023-07-06T20:15:18.322441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:34:49.410663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216730+0000","last_change":"2023-07-06T20:15:16.217727+0000","last_active":"2023-07-06T20:15:18.216730+0000","last_peered":"2023-07-06T20:15:18.216730+0000","last_clean":"2023-07-06T20:15:18.216730+0000","last_became_active":"2023-07-06T20:15:16.217389+0000","last_became_peered":"2023-07-06T20:15:16.217389+0000","last_unstale":"2023-07-06T20:15:18.216730+0000","last_undegraded":"2023-07-06T20:15:18.216730+0000","last_fullsized":"2023-07-06T20:15:18.216730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:07:59.015168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152286+0000","last_change":"2023-07-06T20:15:17.237275+0000","last_active":"2023-07-06T20:15:19.152286+0000","last_peered":"2023-07-06T20:15:19.152286+0000","last_clean":"2023-07-06T20:15:19.152286+0000","last_became_active":"2023-07-06T20:15:17.236862+0000","last_became_peered":"2023-07-06T20:15:17.236862+0000","last_unstale":"2023-07-06T20:15:19.152286+0000","last_undegraded":"2023-07-06T20:15:19.152286+0000","last_fullsized":"2023-07-06T20:15:19.152286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:19:18.047476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216577+0000","last_change":"2023-07-06T20:15:16.218252+0000","last_active":"2023-07-06T20:15:18.216577+0000","last_peered":"2023-07-06T20:15:18.216577+0000","last_clean":"2023-07-06T20:15:18.216577+0000","last_became_active":"2023-07-06T20:15:16.217986+0000","last_became_peered":"2023-07-06T20:15:16.217986+0000","last_unstale":"2023-07-06T20:15:18.216577+0000","last_undegraded":"2023-07-06T20:15:18.216577+0000","last_fullsized":"2023-07-06T20:15:18.216577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:02:56.259164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345352+0000","last_change":"2023-07-06T20:15:17.238880+0000","last_active":"2023-07-06T20:15:18.345352+0000","last_peered":"2023-07-06T20:15:18.345352+0000","last_clean":"2023-07-06T20:15:18.345352+0000","last_became_active":"2023-07-06T20:15:17.238661+0000","last_became_peered":"2023-07-06T20:15:17.238661+0000","last_unstale":"2023-07-06T20:15:18.345352+0000","last_undegraded":"2023-07-06T20:15:18.345352+0000","last_fullsized":"2023-07-06T20:15:18.345352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:30.023080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322862+0000","last_change":"2023-07-06T20:15:16.222750+0000","last_active":"2023-07-06T20:15:18.322862+0000","last_peered":"2023-07-06T20:15:18.322862+0000","last_clean":"2023-07-06T20:15:18.322862+0000","last_became_active":"2023-07-06T20:15:16.222400+0000","last_became_peered":"2023-07-06T20:15:16.222400+0000","last_unstale":"2023-07-06T20:15:18.322862+0000","last_undegraded":"2023-07-06T20:15:18.322862+0000","last_fullsized":"2023-07-06T20:15:18.322862+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:28:59.869660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152191+0000","last_change":"2023-07-06T20:15:17.237349+0000","last_active":"2023-07-06T20:15:19.152191+0000","last_peered":"2023-07-06T20:15:19.152191+0000","last_clean":"2023-07-06T20:15:19.152191+0000","last_became_active":"2023-07-06T20:15:17.236856+0000","last_became_peered":"2023-07-06T20:15:17.236856+0000","last_unstale":"2023-07-06T20:15:19.152191+0000","last_undegraded":"2023-07-06T20:15:19.152191+0000","last_fullsized":"2023-07-06T20:15:19.152191+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:37.253145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216715+0000","last_change":"2023-07-06T20:15:16.217890+0000","last_active":"2023-07-06T20:15:18.216715+0000","last_peered":"2023-07-06T20:15:18.216715+0000","last_clean":"2023-07-06T20:15:18.216715+0000","last_became_active":"2023-07-06T20:15:16.217589+0000","last_became_peered":"2023-07-06T20:15:16.217589+0000","last_unstale":"2023-07-06T20:15:18.216715+0000","last_undegraded":"2023-07-06T20:15:18.216715+0000","last_fullsized":"2023-07-06T20:15:18.216715+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:45:33.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152776+0000","last_change":"2023-07-06T20:15:17.244334+0000","last_active":"2023-07-06T20:15:19.152776+0000","last_peered":"2023-07-06T20:15:19.152776+0000","last_clean":"2023-07-06T20:15:19.152776+0000","last_became_active":"2023-07-06T20:15:17.244066+0000","last_became_peered":"2023-07-06T20:15:17.244066+0000","last_unstale":"2023-07-06T20:15:19.152776+0000","last_undegraded":"2023-07-06T20:15:19.152776+0000","last_fullsized":"2023-07-06T20:15:19.152776+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:54:50.838801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152761+0000","last_change":"2023-07-06T20:15:16.222467+0000","last_active":"2023-07-06T20:15:19.152761+0000","last_peered":"2023-07-06T20:15:19.152761+0000","last_clean":"2023-07-06T20:15:19.152761+0000","last_became_active":"2023-07-06T20:15:16.222147+0000","last_became_peered":"2023-07-06T20:15:16.222147+0000","last_unstale":"2023-07-06T20:15:19.152761+0000","last_undegraded":"2023-07-06T20:15:19.152761+0000","last_fullsized":"2023-07-06T20:15:19.152761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:53:39.180721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701264+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:15:18.701264+0000","last_peered":"2023-07-06T20:15:18.701264+0000","last_clean":"2023-07-06T20:15:18.701264+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:18.701264+0000","last_undegraded":"2023-07-06T20:15:18.701264+0000","last_fullsized":"2023-07-06T20:15:18.701264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:01:20.771894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.151923+0000","last_change":"2023-07-06T20:15:17.233005+0000","last_active":"2023-07-06T20:15:19.151923+0000","last_peered":"2023-07-06T20:15:19.151923+0000","last_clean":"2023-07-06T20:15:19.151923+0000","last_became_active":"2023-07-06T20:15:17.232718+0000","last_became_peered":"2023-07-06T20:15:17.232718+0000","last_unstale":"2023-07-06T20:15:19.151923+0000","last_undegraded":"2023-07-06T20:15:19.151923+0000","last_fullsized":"2023-07-06T20:15:19.151923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:51:28.509267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.152806+0000","last_change":"2023-07-06T20:15:16.214400+0000","last_active":"2023-07-06T20:15:19.152806+0000","last_peered":"2023-07-06T20:15:19.152806+0000","last_clean":"2023-07-06T20:15:19.152806+0000","last_became_active":"2023-07-06T20:15:16.213976+0000","last_became_peered":"2023-07-06T20:15:16.213976+0000","last_unstale":"2023-07-06T20:15:19.152806+0000","last_undegraded":"2023-07-06T20:15:19.152806+0000","last_fullsized":"2023-07-06T20:15:19.152806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:08:33.756544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345545+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:15:18.345545+0000","last_peered":"2023-07-06T20:15:18.345545+0000","last_clean":"2023-07-06T20:15:18.345545+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:18.345545+0000","last_undegraded":"2023-07-06T20:15:18.345545+0000","last_fullsized":"2023-07-06T20:15:18.345545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:25.172680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322885+0000","last_change":"2023-07-06T20:15:17.240556+0000","last_active":"2023-07-06T20:15:18.322885+0000","last_peered":"2023-07-06T20:15:18.322885+0000","last_clean":"2023-07-06T20:15:18.322885+0000","last_became_active":"2023-07-06T20:15:17.240217+0000","last_became_peered":"2023-07-06T20:15:17.240217+0000","last_unstale":"2023-07-06T20:15:18.322885+0000","last_undegraded":"2023-07-06T20:15:18.322885+0000","last_fullsized":"2023-07-06T20:15:18.322885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:19.450885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.344485+0000","last_change":"2023-07-06T20:15:16.218991+0000","last_active":"2023-07-06T20:15:18.344485+0000","last_peered":"2023-07-06T20:15:18.344485+0000","last_clean":"2023-07-06T20:15:18.344485+0000","last_became_active":"2023-07-06T20:15:16.218677+0000","last_became_peered":"2023-07-06T20:15:16.218677+0000","last_unstale":"2023-07-06T20:15:18.344485+0000","last_undegraded":"2023-07-06T20:15:18.344485+0000","last_fullsized":"2023-07-06T20:15:18.344485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:22.712178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.216473+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:15:18.216473+0000","last_peered":"2023-07-06T20:15:18.216473+0000","last_clean":"2023-07-06T20:15:18.216473+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:18.216473+0000","last_undegraded":"2023-07-06T20:15:18.216473+0000","last_fullsized":"2023-07-06T20:15:18.216473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:35:21.741785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345986+0000","last_change":"2023-07-06T20:15:17.233295+0000","last_active":"2023-07-06T20:15:18.345986+0000","last_peered":"2023-07-06T20:15:18.345986+0000","last_clean":"2023-07-06T20:15:18.345986+0000","last_became_active":"2023-07-06T20:15:17.229758+0000","last_became_peered":"2023-07-06T20:15:17.229758+0000","last_unstale":"2023-07-06T20:15:18.345986+0000","last_undegraded":"2023-07-06T20:15:18.345986+0000","last_fullsized":"2023-07-06T20:15:18.345986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:18:28.422068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322773+0000","last_change":"2023-07-06T20:15:16.226751+0000","last_active":"2023-07-06T20:15:18.322773+0000","last_peered":"2023-07-06T20:15:18.322773+0000","last_clean":"2023-07-06T20:15:18.322773+0000","last_became_active":"2023-07-06T20:15:16.226508+0000","last_became_peered":"2023-07-06T20:15:16.226508+0000","last_unstale":"2023-07-06T20:15:18.322773+0000","last_undegraded":"2023-07-06T20:15:18.322773+0000","last_fullsized":"2023-07-06T20:15:18.322773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:03:43.127442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.345491+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:15:18.345491+0000","last_peered":"2023-07-06T20:15:18.345491+0000","last_clean":"2023-07-06T20:15:18.345491+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:18.345491+0000","last_undegraded":"2023-07-06T20:15:18.345491+0000","last_fullsized":"2023-07-06T20:15:18.345491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:30:45.861398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322598+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:15:18.322598+0000","last_peered":"2023-07-06T20:15:18.322598+0000","last_clean":"2023-07-06T20:15:18.322598+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:18.322598+0000","last_undegraded":"2023-07-06T20:15:18.322598+0000","last_fullsized":"2023-07-06T20:15:18.322598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:28:36.054410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701164+0000","last_change":"2023-07-06T20:15:17.227424+0000","last_active":"2023-07-06T20:15:18.701164+0000","last_peered":"2023-07-06T20:15:18.701164+0000","last_clean":"2023-07-06T20:15:18.701164+0000","last_became_active":"2023-07-06T20:15:17.227133+0000","last_became_peered":"2023-07-06T20:15:17.227133+0000","last_unstale":"2023-07-06T20:15:18.701164+0000","last_undegraded":"2023-07-06T20:15:18.701164+0000","last_fullsized":"2023-07-06T20:15:18.701164+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:30.023080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.346074+0000","last_change":"2023-07-06T20:15:16.219749+0000","last_active":"2023-07-06T20:15:18.346074+0000","last_peered":"2023-07-06T20:15:18.346074+0000","last_clean":"2023-07-06T20:15:18.346074+0000","last_became_active":"2023-07-06T20:15:16.219092+0000","last_became_peered":"2023-07-06T20:15:16.219092+0000","last_unstale":"2023-07-06T20:15:18.346074+0000","last_undegraded":"2023-07-06T20:15:18.346074+0000","last_fullsized":"2023-07-06T20:15:18.346074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:03:43.127442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322644+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:15:18.322644+0000","last_peered":"2023-07-06T20:15:18.322644+0000","last_clean":"2023-07-06T20:15:18.322644+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:15:18.322644+0000","last_undegraded":"2023-07-06T20:15:18.322644+0000","last_fullsized":"2023-07-06T20:15:18.322644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:17:04.184086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.700843+0000","last_change":"2023-07-06T20:15:17.240283+0000","last_active":"2023-07-06T20:15:18.700843+0000","last_peered":"2023-07-06T20:15:18.700843+0000","last_clean":"2023-07-06T20:15:18.700843+0000","last_became_active":"2023-07-06T20:15:17.239921+0000","last_became_peered":"2023-07-06T20:15:17.239921+0000","last_unstale":"2023-07-06T20:15:18.700843+0000","last_undegraded":"2023-07-06T20:15:18.700843+0000","last_fullsized":"2023-07-06T20:15:18.700843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:28:39.303552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218334+0000","last_change":"2023-07-06T20:15:16.221737+0000","last_active":"2023-07-06T20:15:18.218334+0000","last_peered":"2023-07-06T20:15:18.218334+0000","last_clean":"2023-07-06T20:15:18.218334+0000","last_became_active":"2023-07-06T20:15:16.221448+0000","last_became_peered":"2023-07-06T20:15:16.221448+0000","last_unstale":"2023-07-06T20:15:18.218334+0000","last_undegraded":"2023-07-06T20:15:18.218334+0000","last_fullsized":"2023-07-06T20:15:18.218334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:49:19.397506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322837+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:15:18.322837+0000","last_peered":"2023-07-06T20:15:18.322837+0000","last_clean":"2023-07-06T20:15:18.322837+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:18.322837+0000","last_undegraded":"2023-07-06T20:15:18.322837+0000","last_fullsized":"2023-07-06T20:15:18.322837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:40:43.891245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453499+0000","last_change":"2023-07-06T20:15:17.228706+0000","last_active":"2023-07-06T20:15:18.453499+0000","last_peered":"2023-07-06T20:15:18.453499+0000","last_clean":"2023-07-06T20:15:18.453499+0000","last_became_active":"2023-07-06T20:15:17.228435+0000","last_became_peered":"2023-07-06T20:15:17.228435+0000","last_unstale":"2023-07-06T20:15:18.453499+0000","last_undegraded":"2023-07-06T20:15:18.453499+0000","last_fullsized":"2023-07-06T20:15:18.453499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:42:08.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.453252+0000","last_change":"2023-07-06T20:15:16.214042+0000","last_active":"2023-07-06T20:15:18.453252+0000","last_peered":"2023-07-06T20:15:18.453252+0000","last_clean":"2023-07-06T20:15:18.453252+0000","last_became_active":"2023-07-06T20:15:16.213603+0000","last_became_peered":"2023-07-06T20:15:16.213603+0000","last_unstale":"2023-07-06T20:15:18.453252+0000","last_undegraded":"2023-07-06T20:15:18.453252+0000","last_fullsized":"2023-07-06T20:15:18.453252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:54:41.418518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.701081+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:15:18.701081+0000","last_peered":"2023-07-06T20:15:18.701081+0000","last_clean":"2023-07-06T20:15:18.701081+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:18.701081+0000","last_undegraded":"2023-07-06T20:15:18.701081+0000","last_fullsized":"2023-07-06T20:15:18.701081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:12:18.106641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.322790+0000","last_change":"2023-07-06T20:15:17.243359+0000","last_active":"2023-07-06T20:15:18.322790+0000","last_peered":"2023-07-06T20:15:18.322790+0000","last_clean":"2023-07-06T20:15:18.322790+0000","last_became_active":"2023-07-06T20:15:17.243081+0000","last_became_peered":"2023-07-06T20:15:17.243081+0000","last_unstale":"2023-07-06T20:15:18.322790+0000","last_undegraded":"2023-07-06T20:15:18.322790+0000","last_fullsized":"2023-07-06T20:15:18.322790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:16.193122+0000","last_clean_scrub_stamp":"2023-07-06T20:15:16.193122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:00.880380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:19.151791+0000","last_change":"2023-07-06T20:15:16.210001+0000","last_active":"2023-07-06T20:15:19.151791+0000","last_peered":"2023-07-06T20:15:19.151791+0000","last_clean":"2023-07-06T20:15:19.151791+0000","last_became_active":"2023-07-06T20:15:16.209479+0000","last_became_peered":"2023-07-06T20:15:16.209479+0000","last_unstale":"2023-07-06T20:15:19.151791+0000","last_undegraded":"2023-07-06T20:15:19.151791+0000","last_fullsized":"2023-07-06T20:15:19.151791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:15.186018+0000","last_clean_scrub_stamp":"2023-07-06T20:15:15.186018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:28:38.296448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-07-06T20:15:18.218535+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:15:18.218535+0000","last_peered":"2023-07-06T20:15:18.218535+0000","last_clean":"2023-07-06T20:15:18.218535+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:18.218535+0000","last_undegraded":"2023-07-06T20:15:18.218535+0000","last_fullsized":"2023-07-06T20:15:18.218535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:40:44.051392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":761722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.81499999999999995}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.64600000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.875}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":170302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.629}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.503}]},{"osd":7,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.307},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.875}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.746}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":752450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.876},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.048}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.089}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":164665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":11.109999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.324999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":11.145,"5min":11.145,"15min":11.145},"last":0.33800000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":11.664,"5min":11.664,"15min":11.664},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":11.208,"5min":11.208,"15min":11.208},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.997999999999999,"5min":10.997999999999999,"15min":10.997999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":11.286,"5min":11.286,"15min":11.286},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.876},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":11.055,"5min":11.055,"15min":11.055},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":11.247999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":170324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.0010000000000003,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.7939999999999996,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.75},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":5.1710000000000003,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.84799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":169832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.624}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:15:22.164 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T20:15:22.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T20:15:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.407+0000 7f5311bff700 1 Processor -- start 2023-07-06T20:15:22.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.408+0000 7f5311bff700 1 -- start start 2023-07-06T20:15:22.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f5311bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f530c15fca0 0x7f530c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f5311bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 0x7f530c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f5311bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0cf780 con 0x7f530c0b5c90 2023-07-06T20:15:22.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f5311bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c0cd490 con 0x7f530c15fca0 2023-07-06T20:15:22.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f5311bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0cd770 con 0x7f530c0b6110 2023-07-06T20:15:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f530bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 0x7f530c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f530bfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 0x7f530c0cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:50804/0 (socket says 172.21.15.27:50804) 2023-07-06T20:15:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f530bfff700 1 -- 172.21.15.27:0/3928937878 learned_addr learned my addr 172.21.15.27:0/3928937878 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.409+0000 7f530b7fe700 1 --2- 172.21.15.27:0/3928937878 >> v2:172.21.15.153:3300/0 conn(0x7f530c15fca0 0x7f530c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530bfff700 1 -- 172.21.15.27:0/3928937878 >> v2:172.21.15.153:3300/0 conn(0x7f530c15fca0 msgr2=0x7f530c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530bfff700 1 --2- 172.21.15.27:0/3928937878 >> v2:172.21.15.153:3300/0 conn(0x7f530c15fca0 0x7f530c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530bfff700 1 -- 172.21.15.27:0/3928937878 >> v1:172.21.15.153:6789/0 conn(0x7f530c0b5c90 legacy=0x7f530c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530bfff700 1 -- 172.21.15.27:0/3928937878 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0cda50 con 0x7f530c0b6110 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530bfff700 1 --2- 172.21.15.27:0/3928937878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 0x7f530c0cfc10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5304009e50 tx=0x7f530400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffad42c01141a58b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530affd700 1 -- 172.21.15.27:0/3928937878 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530400e070 con 0x7f530c0b6110 2023-07-06T20:15:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530affd700 1 -- 172.21.15.27:0/3928937878 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530400b910 con 0x7f530c0b6110 2023-07-06T20:15:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.410+0000 7f530affd700 1 -- 172.21.15.27:0/3928937878 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530400bcc0 con 0x7f530c0b6110 2023-07-06T20:15:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 -- 172.21.15.27:0/3928937878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 msgr2=0x7f530c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 --2- 172.21.15.27:0/3928937878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 0x7f530c0cfc10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5304009e50 tx=0x7f530400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 -- 172.21.15.27:0/3928937878 shutdown_connections 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 --2- 172.21.15.27:0/3928937878 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b6110 0x7f530c0cfc10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 --2- 172.21.15.27:0/3928937878 >> v2:172.21.15.153:3300/0 conn(0x7f530c15fca0 0x7f530c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 -- 172.21.15.27:0/3928937878 >> 172.21.15.27:0/3928937878 conn(0x7f530c06c150 msgr2=0x7f530c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 -- 172.21.15.27:0/3928937878 shutdown_connections 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.411+0000 7f5311bff700 1 -- 172.21.15.27:0/3928937878 wait complete. 2023-07-06T20:15:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.413+0000 7f5311bff700 1 Processor -- start 2023-07-06T20:15:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.413+0000 7f5311bff700 1 -- start start 2023-07-06T20:15:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5311bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b5c90 0x7f530c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5311bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 0x7f530c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5311bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f530c15fca0 0x7f530c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5311bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c14d690 con 0x7f530c0b6110 2023-07-06T20:15:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5311bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c7380 con 0x7f530c15fca0 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5311bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c7660 con 0x7f530c0b5c90 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.414+0000 7f5303fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 0x7f530c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 0x7f530c0c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51556/0 (socket says 172.21.15.27:51556) 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 -- 172.21.15.27:0/360860716 learned_addr learned my addr 172.21.15.27:0/360860716 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f530b7fe700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b5c90 0x7f530c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f530bfff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f530c15fca0 0x7f530c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 -- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b5c90 msgr2=0x7f530c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b5c90 0x7f530c14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 -- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f530c15fca0 msgr2=0x7f530c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f530c15fca0 0x7f530c15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 -- 172.21.15.27:0/360860716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304009b30 con 0x7f530c0b6110 2023-07-06T20:15:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5303fff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 0x7f530c0c6930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c4e0 tx=0x7f52f400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40052d0 con 0x7f530c0b6110 2023-07-06T20:15:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4005430 con 0x7f530c0b6110 2023-07-06T20:15:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.415+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40038c0 con 0x7f530c0b6110 2023-07-06T20:15:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.416+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c0c7940 con 0x7f530c0b6110 2023-07-06T20:15:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.416+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c0c7f00 con 0x7f530c0b6110 2023-07-06T20:15:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.417+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c0d1450 con 0x7f530c0b6110 2023-07-06T20:15:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.418+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52f4003a20 con 0x7f530c0b6110 2023-07-06T20:15:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.418+0000 7f5308ff9700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f52ec06ace0 0x7f52ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.419+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52f40888c0 con 0x7f530c0b6110 2023-07-06T20:15:22.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.419+0000 7f530b7fe700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f52ec06ace0 0x7f52ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.420+0000 7f530b7fe700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f52ec06ace0 0x7f52ec06d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009b40 tx=0x7f52fc009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:22.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.423+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f52f40538d0 con 0x7f530c0b6110 2023-07-06T20:15:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.717+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f530c0cecd0 con 0x7f530c0b6110 2023-07-06T20:15:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.722+0000 7f5308ff9700 1 -- 172.21.15.27:0/360860716 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f52f40170b0 con 0x7f530c0b6110 2023-07-06T20:15:22.724 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-07-06T20:15:22.724 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBGIKdkSs+yFxAABbJ2vpRTPGJa6RY1/Ha9/Q== 2023-07-06T20:15:22.724 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2023-07-06T20:15:22.724 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2023-07-06T20:15:22.724 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T20:15:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f52ec06ace0 msgr2=0x7f52ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f52ec06ace0 0x7f52ec06d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009b40 tx=0x7f52fc009210 comp rx=0 tx=0).stop 2023-07-06T20:15:22.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 msgr2=0x7f530c0c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 0x7f530c0c6930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c4e0 tx=0x7f52f400c7f0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 shutdown_connections 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f52ec06ace0 0x7f52ec06d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f530c15fca0 0x7f530c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f530c0b5c90 0x7f530c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 --2- 172.21.15.27:0/360860716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f530c0b6110 0x7f530c0c6930 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 >> 172.21.15.27:0/360860716 conn(0x7f530c06c150 msgr2=0x7f530c0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:22.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 shutdown_connections 2023-07-06T20:15:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.724+0000 7f5311bff700 1 -- 172.21.15.27:0/360860716 wait complete. 2023-07-06T20:15:22.728 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-07-06T20:15:22.744 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:22.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.945+0000 7f0e605ff700 1 Processor -- start 2023-07-06T20:15:22.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.946+0000 7f0e605ff700 1 -- start start 2023-07-06T20:15:22.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.947+0000 7f0e605ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e5815fca0 0x7f0e580d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.947+0000 7f0e605ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e580cf780 con 0x7f0e580b6110 2023-07-06T20:15:22.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.947+0000 7f0e605ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e580cd490 con 0x7f0e580b5c90 2023-07-06T20:15:22.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.947+0000 7f0e605ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e580cd770 con 0x7f0e5815fca0 2023-07-06T20:15:22.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.947+0000 7f0e5eb9c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e580b6110 0x7f0e580cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49292/0 (socket says 172.21.15.27:49292) 2023-07-06T20:15:22.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.947+0000 7f0e5eb9c700 1 -- 172.21.15.27:0/3357255424 learned_addr learned my addr 172.21.15.27:0/3357255424 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:22.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5e39b700 1 --2- 172.21.15.27:0/3357255424 >> v2:172.21.15.153:3300/0 conn(0x7f0e5815fca0 0x7f0e580d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886968358 0 0) 0x7f0e580cf780 con 0x7f0e580b6110 2023-07-06T20:15:22.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e440035b0 con 0x7f0e580b6110 2023-07-06T20:15:22.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706377719 0 0) 0x7f0e580cd490 con 0x7f0e580b5c90 2023-07-06T20:15:22.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e580cf780 con 0x7f0e580b5c90 2023-07-06T20:15:22.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252437721 0 0) 0x7f0e440035b0 con 0x7f0e580b6110 2023-07-06T20:15:22.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5e39b700 1 -- 172.21.15.27:0/3357255424 >> v1:172.21.15.153:6789/0 conn(0x7f0e580b5c90 legacy=0x7f0e580d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5e39b700 1 -- 172.21.15.27:0/3357255424 >> v1:172.21.15.27:6789/0 conn(0x7f0e580b6110 legacy=0x7f0e580cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5e39b700 1 -- 172.21.15.27:0/3357255424 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580cda50 con 0x7f0e5815fca0 2023-07-06T20:15:22.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.948+0000 7f0e5e39b700 1 --2- 172.21.15.27:0/3357255424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e5815fca0 0x7f0e580d1420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009840 tx=0x7f0e54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22e6953094ff989e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500019f0 con 0x7f0e5815fca0 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54004030 con 0x7f0e5815fca0 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e5d399700 1 -- 172.21.15.27:0/3357255424 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540043c0 con 0x7f0e5815fca0 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 -- 172.21.15.27:0/3357255424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e5815fca0 msgr2=0x7f0e580d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 --2- 172.21.15.27:0/3357255424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e5815fca0 0x7f0e580d1420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009840 tx=0x7f0e54009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 -- 172.21.15.27:0/3357255424 shutdown_connections 2023-07-06T20:15:22.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 --2- 172.21.15.27:0/3357255424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e5815fca0 0x7f0e580d1420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 -- 172.21.15.27:0/3357255424 >> 172.21.15.27:0/3357255424 conn(0x7f0e5806c150 msgr2=0x7f0e5806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:22.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 -- 172.21.15.27:0/3357255424 shutdown_connections 2023-07-06T20:15:22.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.949+0000 7f0e605ff700 1 -- 172.21.15.27:0/3357255424 wait complete. 2023-07-06T20:15:22.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.951+0000 7f0e605ff700 1 Processor -- start 2023-07-06T20:15:22.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.951+0000 7f0e605ff700 1 -- start start 2023-07-06T20:15:22.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e605ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e580b5c90 0x7f0e581f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e605ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e580b6110 0x7f0e58156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e605ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 0x7f0e58158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e605ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e581f1dc0 con 0x7f0e5815fca0 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e605ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580c7be0 con 0x7f0e580b6110 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e605ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580c7ec0 con 0x7f0e580b5c90 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.952+0000 7f0e5eb9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 0x7f0e58158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 0x7f0e58158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51568/0 (socket says 172.21.15.27:51568) 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 -- 172.21.15.27:0/66323629 learned_addr learned my addr 172.21.15.27:0/66323629 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5e39b700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e580b5c90 0x7f0e581f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5db9a700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e580b6110 0x7f0e58156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 -- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e580b5c90 msgr2=0x7f0e581f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e580b5c90 0x7f0e581f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 -- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e580b6110 msgr2=0x7f0e58156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e580b6110 0x7f0e58156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 -- 172.21.15.27:0/66323629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540094a0 con 0x7f0e5815fca0 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.953+0000 7f0e5eb9c700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 0x7f0e58158db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0e50001700 tx=0x7f0e500021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.954+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000ab00 con 0x7f0e5815fca0 2023-07-06T20:15:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.954+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5000ac80 con 0x7f0e5815fca0 2023-07-06T20:15:22.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.954+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e580c81a0 con 0x7f0e5815fca0 2023-07-06T20:15:22.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.954+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500070d0 con 0x7f0e5815fca0 2023-07-06T20:15:22.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.954+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e580c8760 con 0x7f0e5815fca0 2023-07-06T20:15:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.955+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e580c8980 con 0x7f0e5815fca0 2023-07-06T20:15:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.956+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e50007230 con 0x7f0e5815fca0 2023-07-06T20:15:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.957+0000 7f0e4effd700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3c06ace0 0x7f0e3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.957+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e50089370 con 0x7f0e5815fca0 2023-07-06T20:15:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.957+0000 7f0e5e39b700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3c06ace0 0x7f0e3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.958+0000 7f0e5e39b700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3c06ace0 0x7f0e3c06d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0e540090b0 tx=0x7f0e54008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:22.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:22.961+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e50054380 con 0x7f0e5815fca0 2023-07-06T20:15:23.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.214+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e580008d0 con 0x7f0e5815fca0 2023-07-06T20:15:23.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.215+0000 7f0e4effd700 1 -- 172.21.15.27:0/66323629 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f0e500579a0 con 0x7f0e5815fca0 2023-07-06T20:15:23.217 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:23.217 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:15:16.714161+0000","modified":"2023-07-06T20:15:20.237603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/827990525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":827990525},{"type":"v1","addr":"172.21.15.27:6835","nonce":827990525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T20:15:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3c06ace0 msgr2=0x7f0e3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3c06ace0 0x7f0e3c06d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0e540090b0 tx=0x7f0e54008030 comp rx=0 tx=0).stop 2023-07-06T20:15:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 msgr2=0x7f0e58158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 0x7f0e58158db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0e50001700 tx=0x7f0e500021d0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 shutdown_connections 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f0e3c06ace0 0x7f0e3c06d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e5815fca0 0x7f0e58158db0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e580b5c90 0x7f0e581f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 --2- 172.21.15.27:0/66323629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e580b6110 0x7f0e58156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 >> 172.21.15.27:0/66323629 conn(0x7f0e5806c150 msgr2=0x7f0e580d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 shutdown_connections 2023-07-06T20:15:23.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.217+0000 7f0e605ff700 1 -- 172.21.15.27:0/66323629 wait complete. 2023-07-06T20:15:23.221 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2023-07-06T20:15:23.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T20:15:16.714161+0000', 'modified': '2023-07-06T20:15:20.237603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5139}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5139': {'gid': 5139, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.27:6835/827990525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 827990525}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 827990525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T20:15:23.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T20:15:23.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T20:15:23.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.847+0000 7f84247ff700 1 Processor -- start 2023-07-06T20:15:23.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.848+0000 7f84247ff700 1 -- start start 2023-07-06T20:15:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f84247ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f841c15fca0 0x7f841c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f84247ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f841c0b5c90 0x7f841c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f84247ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c0cf780 con 0x7f841c0b6110 2023-07-06T20:15:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f84247ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c0cd490 con 0x7f841c15fca0 2023-07-06T20:15:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f84247ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c0cd770 con 0x7f841c0b5c90 2023-07-06T20:15:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f842259b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f841c15fca0 0x7f841c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f842259b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f841c15fca0 0x7f841c0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51570/0 (socket says 172.21.15.27:51570) 2023-07-06T20:15:23.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.849+0000 7f842259b700 1 -- 172.21.15.27:0/166684666 learned_addr learned my addr 172.21.15.27:0/166684666 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f8421d9a700 1 --2- 172.21.15.27:0/166684666 >> v2:172.21.15.153:3300/0 conn(0x7f841c0b5c90 0x7f841c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f842259b700 1 -- 172.21.15.27:0/166684666 >> v2:172.21.15.153:3300/0 conn(0x7f841c0b5c90 msgr2=0x7f841c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f842259b700 1 --2- 172.21.15.27:0/166684666 >> v2:172.21.15.153:3300/0 conn(0x7f841c0b5c90 0x7f841c0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f842259b700 1 -- 172.21.15.27:0/166684666 >> v1:172.21.15.153:6789/0 conn(0x7f841c0b6110 legacy=0x7f841c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f842259b700 1 -- 172.21.15.27:0/166684666 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c0cda50 con 0x7f841c15fca0 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f842259b700 1 --2- 172.21.15.27:0/166684666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0d1420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f841c06b6f0 tx=0x7f8418009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8ced77aa421e1c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f8421599700 1 -- 172.21.15.27:0/166684666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800b870 con 0x7f841c15fca0 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f8421599700 1 -- 172.21.15.27:0/166684666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800b9d0 con 0x7f841c15fca0 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.850+0000 7f8421599700 1 -- 172.21.15.27:0/166684666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800bd60 con 0x7f841c15fca0 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 -- 172.21.15.27:0/166684666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 msgr2=0x7f841c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 --2- 172.21.15.27:0/166684666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0d1420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f841c06b6f0 tx=0x7f8418009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 -- 172.21.15.27:0/166684666 shutdown_connections 2023-07-06T20:15:23.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 --2- 172.21.15.27:0/166684666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0d1420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 --2- 172.21.15.27:0/166684666 >> v2:172.21.15.153:3300/0 conn(0x7f841c0b5c90 0x7f841c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 -- 172.21.15.27:0/166684666 >> 172.21.15.27:0/166684666 conn(0x7f841c06c150 msgr2=0x7f841c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 -- 172.21.15.27:0/166684666 shutdown_connections 2023-07-06T20:15:23.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.851+0000 7f84247ff700 1 -- 172.21.15.27:0/166684666 wait complete. 2023-07-06T20:15:23.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.853+0000 7f84247ff700 1 Processor -- start 2023-07-06T20:15:23.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.854+0000 7f84247ff700 1 -- start start 2023-07-06T20:15:23.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f84247ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f841c0b5c90 0x7f841c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f84247ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f841c0b6110 0x7f841c156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f84247ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f84247ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0c7fd0 con 0x7f841c15fca0 2023-07-06T20:15:23.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f84247ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0c89a0 con 0x7f841c0b6110 2023-07-06T20:15:23.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f84247ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0c73c0 con 0x7f841c0b5c90 2023-07-06T20:15:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f8422d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f842259b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f841c0b5c90 0x7f841c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f8422d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51580/0 (socket says 172.21.15.27:51580) 2023-07-06T20:15:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f8422d9c700 1 -- 172.21.15.27:0/385316531 learned_addr learned my addr 172.21.15.27:0/385316531 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.855+0000 7f8421d9a700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f841c0b6110 0x7f841c156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8422d9c700 1 -- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f841c0b5c90 msgr2=0x7f841c0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8422d9c700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f841c0b5c90 0x7f841c0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8422d9c700 1 -- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f841c0b6110 msgr2=0x7f841c156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8422d9c700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f841c0b6110 0x7f841c156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8422d9c700 1 -- 172.21.15.27:0/385316531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418009550 con 0x7f841c15fca0 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8422d9c700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0cbc00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f841400cc50 tx=0x7f841400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8412ffd700 1 -- 172.21.15.27:0/385316531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414003c00 con 0x7f841c15fca0 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8412ffd700 1 -- 172.21.15.27:0/385316531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414003d60 con 0x7f841c15fca0 2023-07-06T20:15:23.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f8412ffd700 1 -- 172.21.15.27:0/385316531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841400aab0 con 0x7f841c15fca0 2023-07-06T20:15:23.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c0c76a0 con 0x7f841c15fca0 2023-07-06T20:15:23.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.856+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c0c7c60 con 0x7f841c15fca0 2023-07-06T20:15:23.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.859+0000 7f8412ffd700 1 -- 172.21.15.27:0/385316531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f841400ac10 con 0x7f841c15fca0 2023-07-06T20:15:23.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.860+0000 7f8412ffd700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f840006ace0 0x7f840006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.860+0000 7f8412ffd700 1 -- 172.21.15.27:0/385316531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8414089420 con 0x7f841c15fca0 2023-07-06T20:15:23.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.860+0000 7f842259b700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f840006ace0 0x7f840006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.861+0000 7f842259b700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f840006ace0 0x7f840006d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8418014040 tx=0x7f8418008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.862+0000 7f8438086b80 1 Processor -- start 2023-07-06T20:15:23.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.862+0000 7f8438086b80 1 -- start start 2023-07-06T20:15:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b033255120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03325a190 0x55b033278a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8438086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b033232110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8438086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0331a6ea0 con 0x55b03322fb50 2023-07-06T20:15:23.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b03311a600 con 0x55b033254d50 2023-07-06T20:15:23.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b033189910 con 0x55b03325a190 2023-07-06T20:15:23.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8422d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b033232110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8422d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b033232110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51592/0 (socket says 172.21.15.27:51592) 2023-07-06T20:15:23.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.863+0000 7f8422d9c700 1 -- 172.21.15.27:0/3585902389 learned_addr learned my addr 172.21.15.27:0/3585902389 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f842259b700 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b033255120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8421d9a700 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03325a190 0x55b033278a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8422d9c700 1 -- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03325a190 msgr2=0x55b033278a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8422d9c700 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03325a190 0x55b033278a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8422d9c700 1 -- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 msgr2=0x55b033255120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8422d9c700 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b033255120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8422d9c700 1 -- 172.21.15.27:0/3585902389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b032f76790 con 0x55b03322fb50 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f842259b700 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b033255120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f8422d9c700 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b033232110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f841405ffa0 tx=0x7f8414057bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20d70ad988ec29a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f83fffff700 1 -- 172.21.15.27:0/3585902389 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414056520 con 0x55b03322fb50 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f83fffff700 1 -- 172.21.15.27:0/3585902389 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84140890c0 con 0x55b03322fb50 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.864+0000 7f83fffff700 1 -- 172.21.15.27:0/3585902389 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140578b0 con 0x55b03322fb50 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 -- 172.21.15.27:0/3585902389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 msgr2=0x55b033232110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b033232110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f841405ffa0 tx=0x7f8414057bb0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 -- 172.21.15.27:0/3585902389 shutdown_connections 2023-07-06T20:15:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b033232110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b033255120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 --2- 172.21.15.27:0/3585902389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03325a190 0x55b033278a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 -- 172.21.15.27:0/3585902389 >> 172.21.15.27:0/3585902389 conn(0x55b033227310 msgr2=0x55b0332775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 -- 172.21.15.27:0/3585902389 shutdown_connections 2023-07-06T20:15:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.865+0000 7f8438086b80 1 -- 172.21.15.27:0/3585902389 wait complete. 2023-07-06T20:15:23.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.866+0000 7f8438086b80 1 -- start start 2023-07-06T20:15:23.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.866+0000 7f8438086b80 1 Processor -- start 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b0332303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 0x55b033278f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033279470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033182be0 con 0x55b033254d50 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0331405b0 con 0x55b03325a190 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b033126910 con 0x55b03322fb50 2023-07-06T20:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8438086b80 10 client.0 _ll_register_callbacks cb 0x55b03313eb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8421d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 0x55b033278f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.867+0000 7f8421d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 0x55b033278f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51600/0 (socket says 172.21.15.27:51600) 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 -- 172.21.15.27:0/3674445707 learned_addr learned my addr 172.21.15.27:0/3674445707 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f842259b700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b0332303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8422d9c700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033279470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 -- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 msgr2=0x55b0332303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b0332303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 -- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 msgr2=0x55b033279470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033279470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 -- 172.21.15.27:0/3674445707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c0c7c60 con 0x55b033254d50 2023-07-06T20:15:23.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f8421d9a700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 0x55b033278f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f841c1e9050 tx=0x7f840c0148b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b257e3ae8710854 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f83fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-07-06T20:15:23.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.868+0000 7f83fd7fa700 1 -- 172.21.15.27:0/3674445707 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c004440 con 0x55b033254d50 2023-07-06T20:15:23.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f83fd7fa700 1 -- 172.21.15.27:0/3674445707 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f840c01a030 con 0x55b033254d50 2023-07-06T20:15:23.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b033224920 con 0x55b033254d50 2023-07-06T20:15:23.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f83fd7fa700 1 -- 172.21.15.27:0/3674445707 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f840c014ed0 con 0x55b033254d50 2023-07-06T20:15:23.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 10 client.5304 fetch_fsmap learned FSMap version 16 2023-07-06T20:15:23.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55b0330630a0 con 0x55b033254d50 2023-07-06T20:15:23.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f83fd7fa700 1 -- 172.21.15.27:0/3674445707 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b033224920 con 0x55b033254d50 2023-07-06T20:15:23.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f83fd7fa700 1 -- 172.21.15.27:0/3674445707 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f840c0036a0 con 0x55b033254d50 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 10 client.5304 fetch_fsmap finished waiting for FSMap version 16 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 10 client.5304 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 20 client.5304 populate_metadata read hostname 'smithi027' 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03324d160 0x55b0332940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 4 client.5304 mds_command: new command op to 5139 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.869+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 --> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0332010d0 con 0x55b03324d160 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.870+0000 7f842259b700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03324d160 0x55b0332940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.870+0000 7f842259b700 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03324d160 0x55b0332940a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37354023b537b256 server_cookie=4ff2be65d39a9417 in_seq=0 out_seq=0 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.870+0000 7f83fd7fa700 10 client.5304 ms_handle_connect on v2:172.21.15.27:6833/827990525 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f83fd7fa700 1 -- 172.21.15.27:0/3674445707 <== mds.0 v2:172.21.15.27:6833/827990525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b0332010d0 con 0x55b03324d160 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f83fd7fa700 10 client.5304 handle_command_reply: tid=0 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 client.5304 shutdown 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 msgr2=0x55b033278f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 0x55b033278f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f841c1e9050 tx=0x7f840c0148b0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 shutdown_connections 2023-07-06T20:15:23.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03324d160 0x55b0332940a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033279470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b0332303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f83fd7fa700 0 client.5304 ms_handle_reset on v2:172.21.15.27:6833/827990525 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 --2- 172.21.15.27:0/3674445707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b033254d50 0x55b033278f30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 >> 172.21.15.27:0/3674445707 conn(0x55b033227310 msgr2=0x55b03325a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 shutdown_connections 2023-07-06T20:15:23.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.871+0000 7f8438086b80 1 -- 172.21.15.27:0/3674445707 wait complete. 2023-07-06T20:15:23.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.872+0000 7f8438086b80 20 client.5304 trim_cache size 0 max 16384 2023-07-06T20:15:23.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.890+0000 7f8438086b80 1 Processor -- start 2023-07-06T20:15:23.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.890+0000 7f8438086b80 1 -- start start 2023-07-06T20:15:23.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b03322bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b0332980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8438086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 0x55b033298600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8438086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b032f77390 con 0x55b03325a190 2023-07-06T20:15:23.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0332311d0 con 0x55b033254d50 2023-07-06T20:15:23.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b03324daa0 con 0x55b03322fb50 2023-07-06T20:15:23.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 0x55b033298600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 0x55b033298600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51616/0 (socket says 172.21.15.27:51616) 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 -- 172.21.15.27:0/1749514161 learned_addr learned my addr 172.21.15.27:0/1749514161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f842259b700 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b03322bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8421d9a700 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b0332980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 -- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 msgr2=0x55b03322bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b03322bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 -- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 msgr2=0x55b0332980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b0332980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.891+0000 7f8422d9c700 1 -- 172.21.15.27:0/1749514161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b033148b10 con 0x55b03325a190 2023-07-06T20:15:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8422d9c700 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 0x55b033298600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8414060690 tx=0x7f8414089050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4a1fd6e5b99f70 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8410ff9700 1 -- 172.21.15.27:0/1749514161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414089230 con 0x55b03325a190 2023-07-06T20:15:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8410ff9700 1 -- 172.21.15.27:0/1749514161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414056520 con 0x55b03325a190 2023-07-06T20:15:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8410ff9700 1 -- 172.21.15.27:0/1749514161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140578b0 con 0x55b03325a190 2023-07-06T20:15:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 -- 172.21.15.27:0/1749514161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 msgr2=0x55b033298600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 0x55b033298600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8414060690 tx=0x7f8414089050 comp rx=0 tx=0).stop 2023-07-06T20:15:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 -- 172.21.15.27:0/1749514161 shutdown_connections 2023-07-06T20:15:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03325a190 0x55b033298600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b03322fb50 0x55b03322bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 --2- 172.21.15.27:0/1749514161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b033254d50 0x55b0332980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.892+0000 7f8438086b80 1 -- 172.21.15.27:0/1749514161 >> 172.21.15.27:0/1749514161 conn(0x55b033227310 msgr2=0x55b03322eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.893+0000 7f8438086b80 1 -- 172.21.15.27:0/1749514161 shutdown_connections 2023-07-06T20:15:23.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.893+0000 7f8438086b80 1 -- 172.21.15.27:0/1749514161 wait complete. 2023-07-06T20:15:23.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.893+0000 7f8438086b80 1 -- start start 2023-07-06T20:15:23.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.894+0000 7f8438086b80 1 Processor -- start 2023-07-06T20:15:23.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b0332de540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b033254d50 0x55b0332316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033278bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033271a40 con 0x55b03322fb50 2023-07-06T20:15:23.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b03326ddf0 con 0x55b03325a190 2023-07-06T20:15:23.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b033271230 con 0x55b033254d50 2023-07-06T20:15:23.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f8438086b80 10 client.0 _ll_register_callbacks cb 0x7f840c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:15:23.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.895+0000 7f842259b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b0332de540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b0332de540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51618/0 (socket says 172.21.15.27:51618) 2023-07-06T20:15:23.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 -- 172.21.15.27:0/1661178043 learned_addr learned my addr 172.21.15.27:0/1661178043 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:23.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f8421d9a700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b033254d50 0x55b0332316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f8422d9c700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033278bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 -- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b033254d50 msgr2=0x55b0332316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b033254d50 0x55b0332316d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 -- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 msgr2=0x55b033278bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:23.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033278bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 -- 172.21.15.27:0/1661178043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b032f76790 con 0x55b03322fb50 2023-07-06T20:15:23.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f8422d9c700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033278bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:23.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f842259b700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b0332de540 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8418009130 tx=0x7f84180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf2ec09ed021ab5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:23.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f83fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-07-06T20:15:23.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f83fd7fa700 1 -- 172.21.15.27:0/1661178043 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841802b040 con 0x55b03322fb50 2023-07-06T20:15:23.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.896+0000 7f83fd7fa700 1 -- 172.21.15.27:0/1661178043 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8418034230 con 0x55b03322fb50 2023-07-06T20:15:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84140890c0 con 0x55b03322fb50 2023-07-06T20:15:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f83fd7fa700 1 -- 172.21.15.27:0/1661178043 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f841801f070 con 0x55b03322fb50 2023-07-06T20:15:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 10 client.5310 fetch_fsmap learned FSMap version 16 2023-07-06T20:15:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55b032fd71d0 con 0x55b03322fb50 2023-07-06T20:15:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f83fd7fa700 1 -- 172.21.15.27:0/1661178043 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84140890c0 con 0x55b03322fb50 2023-07-06T20:15:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f83fd7fa700 1 -- 172.21.15.27:0/1661178043 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f84180343f0 con 0x55b03322fb50 2023-07-06T20:15:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 10 client.5310 fetch_fsmap finished waiting for FSMap version 16 2023-07-06T20:15:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 10 client.5310 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T20:15:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 20 client.5310 populate_metadata read hostname 'smithi027' 2023-07-06T20:15:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03326cf90 0x55b03326d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 4 client.5310 mds_command: new command op to 5139 tid=0[{"prefix": "status"}] 2023-07-06T20:15:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 --> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b03317bd20 con 0x55b03326cf90 2023-07-06T20:15:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.897+0000 7f8421d9a700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03326cf90 0x55b03326d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f83fd7fa700 10 client.5310 ms_handle_connect on v2:172.21.15.27:6833/827990525 2023-07-06T20:15:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8421d9a700 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03326cf90 0x55b03326d360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a673fbbe775bf87f server_cookie=5be10b7fb7f137d8 in_seq=0 out_seq=0 2023-07-06T20:15:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f83fd7fa700 1 -- 172.21.15.27:0/1661178043 <== mds.0 v2:172.21.15.27:6833/827990525 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b03317bd20 con 0x55b03326cf90 2023-07-06T20:15:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f83fd7fa700 10 client.5310 handle_command_reply: tid=0 2023-07-06T20:15:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 client.5310 shutdown 2023-07-06T20:15:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 msgr2=0x55b0332de540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b0332de540 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8418009130 tx=0x7f84180047c0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 shutdown_connections 2023-07-06T20:15:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:6833/827990525,v1:172.21.15.27:6835/827990525] conn(0x55b03326cf90 0x55b03326d360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b03325a190 0x55b033278bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b03322fb50 0x55b0332de540 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 --2- 172.21.15.27:0/1661178043 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55b033254d50 0x55b0332316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 >> 172.21.15.27:0/1661178043 conn(0x55b033227310 msgr2=0x55b03324b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.898+0000 7f83fd7fa700 0 client.5310 ms_handle_reset on v2:172.21.15.27:6833/827990525 2023-07-06T20:15:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 shutdown_connections 2023-07-06T20:15:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f8438086b80 1 -- 172.21.15.27:0/1661178043 wait complete. 2023-07-06T20:15:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f8438086b80 20 client.5310 trim_cache size 0 max 16384 2023-07-06T20:15:23.917 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c2a9da8e-f30b-43f7-b235-3303ae838054", 2023-07-06T20:15:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-07-06T20:15:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5139, 2023-07-06T20:15:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-07-06T20:15:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-07-06T20:15:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-07-06T20:15:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 6.6713295490000002, 2023-07-06T20:15:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 16, 2023-07-06T20:15:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 36, 2023-07-06T20:15:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 36, 2023-07-06T20:15:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 15.560909335 2023-07-06T20:15:23.919 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:23.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f840006ace0 msgr2=0x7f840006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f840006ace0 0x7f840006d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8418014040 tx=0x7f8418008030 comp rx=0 tx=0).stop 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 msgr2=0x7f841c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0cbc00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f841400cc50 tx=0x7f841400cf60 comp rx=0 tx=0).stop 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 shutdown_connections 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f840006ace0 0x7f840006d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f841c15fca0 0x7f841c0cbc00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f841c0b5c90 0x7f841c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 --2- 172.21.15.27:0/385316531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f841c0b6110 0x7f841c156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 >> 172.21.15.27:0/385316531 conn(0x7f841c06c150 msgr2=0x7f841c073060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.899+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 shutdown_connections 2023-07-06T20:15:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:23.900+0000 7f84247ff700 1 -- 172.21.15.27:0/385316531 wait complete. 2023-07-06T20:15:23.922 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T20:15:24.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.124+0000 7fcb8a9ff700 1 Processor -- start 2023-07-06T20:15:24.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.124+0000 7fcb8a9ff700 1 -- start start 2023-07-06T20:15:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.125+0000 7fcb8a9ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb8415fca0 0x7fcb840d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.125+0000 7fcb8a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.125+0000 7fcb8a9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb840cf780 con 0x7fcb840b60e0 2023-07-06T20:15:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.125+0000 7fcb8a9ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb840cd490 con 0x7fcb8415fca0 2023-07-06T20:15:24.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.125+0000 7fcb8a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840cd770 con 0x7fcb840b5c90 2023-07-06T20:15:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb8415fca0 0x7fcb840d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb837fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:24.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb8415fca0 0x7fcb840d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:54122/0 (socket says 172.21.15.27:54122) 2023-07-06T20:15:24.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 -- 172.21.15.27:0/4198652432 learned_addr learned my addr 172.21.15.27:0/4198652432 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:24.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb88f9c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcb840b60e0 0x7fcb840cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:50218/0 (socket says 172.21.15.27:50218) 2023-07-06T20:15:24.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 -- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 msgr2=0x7fcb840d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 --2- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 -- 172.21.15.27:0/4198652432 >> v1:172.21.15.153:6789/0 conn(0x7fcb840b60e0 legacy=0x7fcb840cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 -- 172.21.15.27:0/4198652432 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840cda50 con 0x7fcb8415fca0 2023-07-06T20:15:24.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.126+0000 7fcb83fff700 1 --2- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb8415fca0 0x7fcb840d1420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb740098f0 tx=0x7fcb74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24132aeb3c81513b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb82ffd700 1 -- 172.21.15.27:0/4198652432 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74011830 con 0x7fcb8415fca0 2023-07-06T20:15:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb82ffd700 1 -- 172.21.15.27:0/4198652432 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb74011990 con 0x7fcb8415fca0 2023-07-06T20:15:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb82ffd700 1 -- 172.21.15.27:0/4198652432 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74011d20 con 0x7fcb8415fca0 2023-07-06T20:15:24.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb8415fca0 msgr2=0x7fcb840d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb8415fca0 0x7fcb840d1420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb740098f0 tx=0x7fcb74009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/4198652432 shutdown_connections 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb8415fca0 0x7fcb840d1420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/4198652432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/4198652432 >> 172.21.15.27:0/4198652432 conn(0x7fcb8406c150 msgr2=0x7fcb8406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/4198652432 shutdown_connections 2023-07-06T20:15:24.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.127+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/4198652432 wait complete. 2023-07-06T20:15:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.129+0000 7fcb8a9ff700 1 Processor -- start 2023-07-06T20:15:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.129+0000 7fcb8a9ff700 1 -- start start 2023-07-06T20:15:24.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.130+0000 7fcb8a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.130+0000 7fcb8a9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb840b60e0 0x7fcb8414d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.130+0000 7fcb8a9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 0x7fcb840b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.130+0000 7fcb8a9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840cbf00 con 0x7fcb8415fca0 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.130+0000 7fcb8a9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840cc3c0 con 0x7fcb840b60e0 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.130+0000 7fcb8a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840a99d0 con 0x7fcb840b5c90 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 0x7fcb840b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 0x7fcb840b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51630/0 (socket says 172.21.15.27:51630) 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 -- 172.21.15.27:0/3385797213 learned_addr learned my addr 172.21.15.27:0/3385797213 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb837fe700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb840b60e0 0x7fcb8414d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb83fff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 -- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 msgr2=0x7fcb840cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 -- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb840b60e0 msgr2=0x7fcb8414d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb840b60e0 0x7fcb8414d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 -- 172.21.15.27:0/3385797213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74009550 con 0x7fcb8415fca0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.131+0000 7fcb88f9c700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 0x7fcb840b7570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00cc00 tx=0x7fcb7c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.132+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00aa00 con 0x7fcb8415fca0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.132+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00ab60 con 0x7fcb8415fca0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.132+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c007070 con 0x7fcb8415fca0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.132+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8415bf00 con 0x7fcb8415fca0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.132+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8415c4c0 con 0x7fcb8415fca0 2023-07-06T20:15:24.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.133+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb840a9ba0 con 0x7fcb8415fca0 2023-07-06T20:15:24.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.134+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb7c0071d0 con 0x7fcb8415fca0 2023-07-06T20:15:24.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.135+0000 7fcb80ff9700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcb7006ace0 0x7fcb7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:24.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.135+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb7c089090 con 0x7fcb8415fca0 2023-07-06T20:15:24.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.135+0000 7fcb83fff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcb7006ace0 0x7fcb7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:24.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.136+0000 7fcb83fff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcb7006ace0 0x7fcb7006d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb74014040 tx=0x7fcb74008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:24.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.139+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb840a9ba0 con 0x7fcb8415fca0 2023-07-06T20:15:24.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.391+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb840a9ba0 con 0x7fcb8415fca0 2023-07-06T20:15:24.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.391+0000 7fcb80ff9700 1 -- 172.21.15.27:0/3385797213 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb840a9ba0 con 0x7fcb8415fca0 2023-07-06T20:15:24.393 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:15:24.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.393+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcb7006ace0 msgr2=0x7fcb7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.393+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcb7006ace0 0x7fcb7006d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb74014040 tx=0x7fcb74008030 comp rx=0 tx=0).stop 2023-07-06T20:15:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.393+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 msgr2=0x7fcb840b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 0x7fcb840b7570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00cc00 tx=0x7fcb7c00cf10 comp rx=0 tx=0).stop 2023-07-06T20:15:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 shutdown_connections 2023-07-06T20:15:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fcb7006ace0 0x7fcb7006d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8415fca0 0x7fcb840b7570 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcb840b5c90 0x7fcb840cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 --2- 172.21.15.27:0/3385797213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb840b60e0 0x7fcb8414d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 >> 172.21.15.27:0/3385797213 conn(0x7fcb8406c150 msgr2=0x7fcb840d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 shutdown_connections 2023-07-06T20:15:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:24.394+0000 7fcb8a9ff700 1 -- 172.21.15.27:0/3385797213 wait complete. 2023-07-06T20:15:24.414 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-06T20:15:24.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:15:24.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-07-06T20:15:24.414 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T20:15:24.415 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T20:15:24.415 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-06T20:15:24.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-06T20:15:24.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:15:24.416 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T20:15:24.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.416 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-07-06T20:15:24.464 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:15:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:15:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-07-06T20:15:24.465 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41885sec preferred_lft 41885sec 2023-07-06T20:15:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 5a:2d:cf:09:4f:ea brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T20:15:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::a0e5:ddff:fe7a:3406/64 scope link 2023-07-06T20:15:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.468 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 5e:d6:af:22:e7:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::5cd6:afff:fe22:e78c/64 scope link 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 5a:2d:cf:09:4f:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::582d:cfff:fe09:4fea/64 scope link 2023-07-06T20:15:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:24.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.471 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:15:24.540 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-07-06T20:15:24.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.543 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-07-06T20:15:24.592 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-07-06T20:15:24.592 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:15:24.593 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-06T20:15:24.593 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-06T20:15:24.593 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:15:24.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.594 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-07-06T20:15:24.595 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-07-06T20:15:24.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-06T20:15:24.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-06T20:15:24.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-06T20:15:24.596 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-07-06T20:15:24.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.769 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:15:24.815 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T20:15:24.815 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T20:15:24.816 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T20:15:24.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-07-06T20:15:24.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:15:24.889 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T20:15:24.935 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T20:15:24.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:15:24.936 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T20:15:24.982 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-06T20:15:24.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:24.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:15:25.129 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:15:25.130 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:15:25.157 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:15:25.158 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:15:25.158 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.153:3300,172.21.15.153:3301" 2023-07-06T20:15:25.158 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@c2a9da8e-f30b-43f7-b235-3303ae838054.cephfs=/ 2023-07-06T20:15:25.158 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.153:3300/172.21.15.153:3301" will pass to kernel 2023-07-06T20:15:25.162 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:15:25.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:25.163 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T20:15:25.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:25.236 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:15:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:client4985 2023-07-06T20:15:25.341 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:15:25.341 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:25.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T20:15:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.629+0000 7ff1f2bff700 1 Processor -- start 2023-07-06T20:15:25.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.629+0000 7ff1f2bff700 1 -- start start 2023-07-06T20:15:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f2bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 0x7ff1ec0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f2bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0cf780 con 0x7ff1ec0b6100 2023-07-06T20:15:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f2bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0cd490 con 0x7ff1ec0b5c90 2023-07-06T20:15:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f2bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0cd770 con 0x7ff1ec15fca0 2023-07-06T20:15:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f119c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff1ec0b6100 0x7ff1ec0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49304/0 (socket says 172.21.15.27:49304) 2023-07-06T20:15:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f119c700 1 -- 172.21.15.27:0/784278669 learned_addr learned my addr 172.21.15.27:0/784278669 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1f099b700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 0x7ff1ec0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.631+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065299699 0 0) 0x7ff1ec0cf780 con 0x7ff1ec0b6100 2023-07-06T20:15:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1dc0035b0 con 0x7ff1ec0b6100 2023-07-06T20:15:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 467613755 0 0) 0x7ff1dc0035b0 con 0x7ff1ec0b6100 2023-07-06T20:15:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 msgr2=0x7ff1ec0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 0x7ff1ec0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 >> v1:172.21.15.153:6789/0 conn(0x7ff1ec0b5c90 legacy=0x7ff1ec0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0cda50 con 0x7ff1ec0b6100 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7ff1e40019f0 con 0x7ff1ec0b6100 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 >> v1:172.21.15.27:6789/0 conn(0x7ff1ec0b6100 legacy=0x7ff1ec0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc003a30 0x7ff1dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0043d0 0x7ff1dc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 0x7ff1dc0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0cd770 con 0x7ff1dc003a30 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc00ba10 con 0x7ff1dc0043d0 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc00bba0 con 0x7ff1ec15fca0 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.632+0000 7ff1f099b700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc003a30 0x7ff1dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f119c700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 0x7ff1dc0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1ebfff700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0043d0 0x7ff1dc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f099b700 1 -- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 msgr2=0x7ff1dc0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f099b700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec15fca0 0x7ff1dc0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f099b700 1 -- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0043d0 msgr2=0x7ff1dc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f099b700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0043d0 0x7ff1dc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f099b700 1 -- 172.21.15.27:0/784278669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0035b0 con 0x7ff1dc003a30 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1ebfff700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc0043d0 0x7ff1dc004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1f099b700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc003a30 0x7ff1dc003e00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009b60 tx=0x7ff1dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ac1482ca3f683f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000e6c0 con 0x7ff1dc003a30 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e000e820 con 0x7ff1dc003a30 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.633+0000 7ff1eb7fe700 1 -- 172.21.15.27:0/784278669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000eb90 con 0x7ff1dc003a30 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f2bff700 1 -- 172.21.15.27:0/784278669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc003a30 msgr2=0x7ff1dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/784278669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1dc003a30 0x7ff1dc003e00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009b60 tx=0x7ff1dc005880 comp rx=0 tx=0).stop 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f119c700 1 -- 172.21.15.27:0/784278669 reap_dead start 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f2bff700 1 -- 172.21.15.27:0/784278669 shutdown_connections 2023-07-06T20:15:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f2bff700 1 -- 172.21.15.27:0/784278669 >> 172.21.15.27:0/784278669 conn(0x7ff1ec06c150 msgr2=0x7ff1ec06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f2bff700 1 -- 172.21.15.27:0/784278669 shutdown_connections 2023-07-06T20:15:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.634+0000 7ff1f2bff700 1 -- 172.21.15.27:0/784278669 wait complete. 2023-07-06T20:15:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.636+0000 7ff1f2bff700 1 Processor -- start 2023-07-06T20:15:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.636+0000 7ff1f2bff700 1 -- start start 2023-07-06T20:15:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f2bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc003a30 0x7ff1ec0c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f2bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec0b8910 0x7ff1ec151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f2bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 0x7ff1ec1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f2bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1f5f00 con 0x7ff1ec1f5550 2023-07-06T20:15:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f2bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0cb220 con 0x7ff1dc003a30 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f2bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec151990 con 0x7ff1ec0b8910 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 0x7ff1ec1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 0x7ff1ec1f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51648/0 (socket says 172.21.15.27:51648) 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 -- 172.21.15.27:0/3869684601 learned_addr learned my addr 172.21.15.27:0/3869684601 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 -- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec0b8910 msgr2=0x7ff1ec151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec0b8910 0x7ff1ec151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 -- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc003a30 msgr2=0x7ff1ec0c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f099b700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc003a30 0x7ff1ec0c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc003a30 0x7ff1ec0c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.637+0000 7ff1f119c700 1 -- 172.21.15.27:0/3869684601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0b8d80 con 0x7ff1ec1f5550 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.638+0000 7ff1f119c700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 0x7ff1ec1f59c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0d3ed0 tx=0x7ff1e4007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.638+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400eba0 con 0x7ff1ec1f5550 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.638+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400ed00 con 0x7ff1ec1f5550 2023-07-06T20:15:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.638+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4013660 con 0x7ff1ec1f5550 2023-07-06T20:15:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.638+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0ca610 con 0x7ff1ec1f5550 2023-07-06T20:15:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.638+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0cabd0 con 0x7ff1ec1f5550 2023-07-06T20:15:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.639+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec0008d0 con 0x7ff1ec1f5550 2023-07-06T20:15:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.640+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1e40137c0 con 0x7ff1ec1f5550 2023-07-06T20:15:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.642+0000 7ff1e97fa700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff1d006ad10 0x7ff1d006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.642+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1e40899f0 con 0x7ff1ec1f5550 2023-07-06T20:15:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.642+0000 7ff1f099b700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff1d006ad10 0x7ff1d006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:25.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.643+0000 7ff1f099b700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff1d006ad10 0x7ff1d006d1c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0000c00 tx=0x7ff1e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.646+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff1e4054a00 con 0x7ff1ec1f5550 2023-07-06T20:15:25.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.898+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1ec0cb360 con 0x7ff1ec1f5550 2023-07-06T20:15:25.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.898+0000 7ff1e97fa700 1 -- 172.21.15.27:0/3869684601 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff1e4058020 con 0x7ff1ec1f5550 2023-07-06T20:15:25.900 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:15:25.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff1d006ad10 msgr2=0x7ff1d006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff1d006ad10 0x7ff1d006d1c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0000c00 tx=0x7ff1e0008030 comp rx=0 tx=0).stop 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 msgr2=0x7ff1ec1f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 0x7ff1ec1f59c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0d3ed0 tx=0x7ff1e4007c90 comp rx=0 tx=0).stop 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 shutdown_connections 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff1d006ad10 0x7ff1d006d1c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ec1f5550 0x7ff1ec1f59c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1dc003a30 0x7ff1ec0c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 --2- 172.21.15.27:0/3869684601 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff1ec0b8910 0x7ff1ec151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 >> 172.21.15.27:0/3869684601 conn(0x7ff1ec06c150 msgr2=0x7ff1ec15a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 shutdown_connections 2023-07-06T20:15:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:25.900+0000 7ff1f2bff700 1 -- 172.21.15.27:0/3869684601 wait complete. 2023-07-06T20:15:25.920 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-06T20:15:25.920 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:15:25.921 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-07-06T20:15:25.921 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T20:15:25.921 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T20:15:25.922 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-06T20:15:25.922 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-06T20:15:25.922 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:15:25.923 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T20:15:25.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:25.924 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-07-06T20:15:25.977 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:15:25.977 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:15:25.977 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:15:25.977 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.977 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2023-07-06T20:15:25.978 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 41775sec preferred_lft 41775sec 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2023-07-06T20:15:25.979 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.980 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T20:15:25.980 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 5e:c1:00:0d:90:58 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:15:25.980 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T20:15:25.980 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::90a5:cff:fe20:b613/64 scope link 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether d2:d2:ab:a1:94:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::d0d2:abff:fea1:94c6/64 scope link 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.981 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T20:15:25.982 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 5e:c1:00:0d:90:58 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T20:15:25.982 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::5cc1:ff:fe0d:9058/64 scope link 2023-07-06T20:15:25.982 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:15:25.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:25.983 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:15:26.054 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-07-06T20:15:26.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.058 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-07-06T20:15:26.109 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-07-06T20:15:26.109 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:15:26.110 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T20:15:26.110 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T20:15:26.110 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:15:26.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.111 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-07-06T20:15:26.112 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-07-06T20:15:26.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T20:15:26.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T20:15:26.114 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T20:15:26.114 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-07-06T20:15:26.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.283 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:15:26.329 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T20:15:26.329 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T20:15:26.330 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T20:15:26.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-07-06T20:15:26.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:15:26.440 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T20:15:26.485 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T20:15:26.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:15:26.486 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T20:15:26.570 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-07-06T20:15:26.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.571 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:15:26.714 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:15:26.714 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:15:26.742 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T20:15:26.742 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T20:15:26.743 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.153:3300,172.21.15.153:3301" 2023-07-06T20:15:26.743 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with new device syntax: 1@c2a9da8e-f30b-43f7-b235-3303ae838054.cephfs=/ 2023-07-06T20:15:26.743 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.153:3300/172.21.15.153:3301" will pass to kernel 2023-07-06T20:15:26.747 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:15:26.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.747 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T20:15:26.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.817 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:15:26.904 INFO:teuthology.orchestra.run.smithi153.stdout:client4991 2023-07-06T20:15:26.905 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:15:26.905 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:26.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:26.951 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-07-06T20:15:27.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:27.010 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 0 ./subdir) 2023-07-06T20:15:27.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:27.074 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes ./subdir) 2023-07-06T20:15:27.121 INFO:teuthology.orchestra.run.smithi027.stderr:./subdir: ceph.quota.max_bytes: No such attribute 2023-07-06T20:15:27.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:15:27.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:27.123 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir) 2023-07-06T20:15:27.169 INFO:teuthology.orchestra.run.smithi027.stderr:setfattr: ./subdir: Invalid argument 2023-07-06T20:15:27.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:15:27.170 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T20:15:27.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T20:15:27.270 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T20:15:27.371 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T20:15:27.372 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:15:27.372 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:27.419 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:15:27.420 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:27.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:27.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.747+0000 7f6370fff700 1 Processor -- start 2023-07-06T20:15:27.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.748+0000 7f6370fff700 1 -- start start 2023-07-06T20:15:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f6370fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f636c15df60 0x7f636c070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f6370fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636c1441f0 con 0x7f636c143900 2023-07-06T20:15:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f6370fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636c15e3e0 con 0x7f636c0b0440 2023-07-06T20:15:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f6370fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636c15a930 con 0x7f636c15df60 2023-07-06T20:15:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f6363fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f636c143900 0x7f636c15b0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49314/0 (socket says 172.21.15.27:49314) 2023-07-06T20:15:27.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f6363fff700 1 -- 172.21.15.27:0/289840836 learned_addr learned my addr 172.21.15.27:0/289840836 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:27.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.749+0000 7f636ad9d700 1 --2- 172.21.15.27:0/289840836 >> v2:172.21.15.153:3300/0 conn(0x7f636c15df60 0x7f636c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:27.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14664959 0 0) 0x7f636c1441f0 con 0x7f636c143900 2023-07-06T20:15:27.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63580035b0 con 0x7f636c143900 2023-07-06T20:15:27.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137795611 0 0) 0x7f636c15e3e0 con 0x7f636c0b0440 2023-07-06T20:15:27.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f636c1441f0 con 0x7f636c0b0440 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f636ad9d700 1 -- 172.21.15.27:0/289840836 >> v1:172.21.15.153:6789/0 conn(0x7f636c0b0440 legacy=0x7f636c0b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f636ad9d700 1 -- 172.21.15.27:0/289840836 >> v1:172.21.15.27:6789/0 conn(0x7f636c143900 legacy=0x7f636c15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f636ad9d700 1 -- 172.21.15.27:0/289840836 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c15e660 con 0x7f636c15df60 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f636ad9d700 1 --2- 172.21.15.27:0/289840836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c070660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6364009d90 tx=0x7f636400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8604e88740946dc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400e070 con 0x7f636c15df60 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636400bb20 con 0x7f636c15df60 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.750+0000 7f6369d9b700 1 -- 172.21.15.27:0/289840836 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364012430 con 0x7f636c15df60 2023-07-06T20:15:27.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 -- 172.21.15.27:0/289840836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 msgr2=0x7f636c070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 --2- 172.21.15.27:0/289840836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c070660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6364009d90 tx=0x7f636400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:27.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 -- 172.21.15.27:0/289840836 shutdown_connections 2023-07-06T20:15:27.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 --2- 172.21.15.27:0/289840836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c070660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 -- 172.21.15.27:0/289840836 >> 172.21.15.27:0/289840836 conn(0x7f636c0cb440 msgr2=0x7f636c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:27.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 -- 172.21.15.27:0/289840836 shutdown_connections 2023-07-06T20:15:27.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.751+0000 7f6370fff700 1 -- 172.21.15.27:0/289840836 wait complete. 2023-07-06T20:15:27.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.753+0000 7f6370fff700 1 Processor -- start 2023-07-06T20:15:27.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.753+0000 7f6370fff700 1 -- start start 2023-07-06T20:15:27.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6370fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f636c0b0440 0x7f636c0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:27.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6370fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 0x7f636c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:27.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6370fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c1f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:27.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6370fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0ccfa0 con 0x7f636c143900 2023-07-06T20:15:27.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6370fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0af490 con 0x7f636c15df60 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6370fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0cd1f0 con 0x7f636c0b0440 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6363fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 0x7f636c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6363fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 0x7f636c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51660/0 (socket says 172.21.15.27:51660) 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.754+0000 7f6363fff700 1 -- 172.21.15.27:0/4159628490 learned_addr learned my addr 172.21.15.27:0/4159628490 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f636a59c700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f636c0b0440 0x7f636c0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f636ad9d700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c1f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6363fff700 1 -- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f636c0b0440 msgr2=0x7f636c0aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6363fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f636c0b0440 0x7f636c0aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6363fff700 1 -- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 msgr2=0x7f636c1f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6363fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c1f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6363fff700 1 -- 172.21.15.27:0/4159628490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364009a70 con 0x7f636c143900 2023-07-06T20:15:27.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f636a59c700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f636c0b0440 0x7f636c0aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:27.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6363fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 0x7f636c0cdfb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6354001eb0 tx=0x7f635400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:27.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354005d70 con 0x7f636c143900 2023-07-06T20:15:27.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63540070d0 con 0x7f636c143900 2023-07-06T20:15:27.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354007460 con 0x7f636c143900 2023-07-06T20:15:27.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.755+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c0cd4d0 con 0x7f636c143900 2023-07-06T20:15:27.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.756+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c0cda90 con 0x7f636c143900 2023-07-06T20:15:27.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.757+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c06fa50 con 0x7f636c143900 2023-07-06T20:15:27.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.758+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63540075c0 con 0x7f636c143900 2023-07-06T20:15:27.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.759+0000 7f6362ffd700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f634c06ace0 0x7f634c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:27.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.759+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f63540889f0 con 0x7f636c143900 2023-07-06T20:15:27.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.760+0000 7f636a59c700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f634c06ace0 0x7f634c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:27.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.760+0000 7f636a59c700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f634c06ace0 0x7f634c06d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f635c0011e0 tx=0x7f635c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.764+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6354053a00 con 0x7f636c143900 2023-07-06T20:15:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.974+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f636c0cdcb0 con 0x7f636c143900 2023-07-06T20:15:27.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.976+0000 7f6362ffd700 1 -- 172.21.15.27:0/4159628490 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f6354080e60 con 0x7f636c143900 2023-07-06T20:15:27.978 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:27.978 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:18.212485+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:27.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f634c06ace0 msgr2=0x7f634c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f634c06ace0 0x7f634c06d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f635c0011e0 tx=0x7f635c009040 comp rx=0 tx=0).stop 2023-07-06T20:15:27.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 msgr2=0x7f636c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:27.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 0x7f636c0cdfb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6354001eb0 tx=0x7f635400c7b0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 shutdown_connections 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f634c06ace0 0x7f634c06d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f636c15df60 0x7f636c1f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f636c0b0440 0x7f636c0aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 --2- 172.21.15.27:0/4159628490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c143900 0x7f636c0cdfb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 >> 172.21.15.27:0/4159628490 conn(0x7f636c0cb440 msgr2=0x7f636c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:27.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 shutdown_connections 2023-07-06T20:15:27.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:27.978+0000 7f6370fff700 1 -- 172.21.15.27:0/4159628490 wait complete. 2023-07-06T20:15:27.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:15:15.163204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:15:15.707733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:15:27.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:15:28.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.204+0000 7f279a9ff700 1 Processor -- start 2023-07-06T20:15:28.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.204+0000 7f279a9ff700 1 -- start start 2023-07-06T20:15:28.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f279a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f279415fca0 0x7f27940d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f279a9ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27940cf780 con 0x7f27940b6100 2023-07-06T20:15:28.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f279a9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27940cd490 con 0x7f27940b5c90 2023-07-06T20:15:28.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f279a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27940cd770 con 0x7f279415fca0 2023-07-06T20:15:28.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f2798f9c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f27940b6100 0x7f27940cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49320/0 (socket says 172.21.15.27:49320) 2023-07-06T20:15:28.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f2798f9c700 1 -- 172.21.15.27:0/1363695876 learned_addr learned my addr 172.21.15.27:0/1363695876 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:28.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.206+0000 7f2793fff700 1 --2- 172.21.15.27:0/1363695876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f279415fca0 0x7f27940d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652791392 0 0) 0x7f27940cf780 con 0x7f27940b6100 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f277c0035b0 con 0x7f27940b6100 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207223241 0 0) 0x7f27940cd490 con 0x7f27940b5c90 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27940cf780 con 0x7f27940b5c90 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2793fff700 1 -- 172.21.15.27:0/1363695876 >> v1:172.21.15.153:6789/0 conn(0x7f27940b5c90 legacy=0x7f27940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2793fff700 1 -- 172.21.15.27:0/1363695876 >> v1:172.21.15.27:6789/0 conn(0x7f27940b6100 legacy=0x7f27940cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2793fff700 1 -- 172.21.15.27:0/1363695876 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940cda50 con 0x7f279415fca0 2023-07-06T20:15:28.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2793fff700 1 --2- 172.21.15.27:0/1363695876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f279415fca0 0x7f27940d1420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2784009870 tx=0x7f2784009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5588ca8cb9321218 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:28.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784011830 con 0x7f279415fca0 2023-07-06T20:15:28.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784011990 con 0x7f279415fca0 2023-07-06T20:15:28.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.207+0000 7f2792ffd700 1 -- 172.21.15.27:0/1363695876 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784011d20 con 0x7f279415fca0 2023-07-06T20:15:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 -- 172.21.15.27:0/1363695876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f279415fca0 msgr2=0x7f27940d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 --2- 172.21.15.27:0/1363695876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f279415fca0 0x7f27940d1420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2784009870 tx=0x7f2784009b80 comp rx=0 tx=0).stop 2023-07-06T20:15:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 -- 172.21.15.27:0/1363695876 shutdown_connections 2023-07-06T20:15:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 --2- 172.21.15.27:0/1363695876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f279415fca0 0x7f27940d1420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 -- 172.21.15.27:0/1363695876 >> 172.21.15.27:0/1363695876 conn(0x7f279406c150 msgr2=0x7f279406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 -- 172.21.15.27:0/1363695876 shutdown_connections 2023-07-06T20:15:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.208+0000 7f279a9ff700 1 -- 172.21.15.27:0/1363695876 wait complete. 2023-07-06T20:15:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.210+0000 7f279a9ff700 1 Processor -- start 2023-07-06T20:15:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.210+0000 7f279a9ff700 1 -- start start 2023-07-06T20:15:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f279a9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f27940b5c90 0x7f27940cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f279a9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 0x7f27940b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f279a9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f279415fca0 0x7f2794153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f279a9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794159be0 con 0x7f27940b6100 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f279a9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794157c00 con 0x7f279415fca0 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f279a9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279415a450 con 0x7f27940b5c90 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.211+0000 7f27937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 0x7f27940b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 0x7f27940b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51672/0 (socket says 172.21.15.27:51672) 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 -- 172.21.15.27:0/3160625258 learned_addr learned my addr 172.21.15.27:0/3160625258 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f2798f9c700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f279415fca0 0x7f2794153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f2793fff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f27940b5c90 0x7f27940cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 -- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f27940b5c90 msgr2=0x7f27940cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f27940b5c90 0x7f27940cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 -- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f279415fca0 msgr2=0x7f2794153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f279415fca0 0x7f2794153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 -- 172.21.15.27:0/3160625258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840094a0 con 0x7f27940b6100 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f27937fe700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 0x7f27940b3150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f278800c520 tx=0x7f278800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.212+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880052d0 con 0x7f27940b6100 2023-07-06T20:15:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.213+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788005430 con 0x7f27940b6100 2023-07-06T20:15:28.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.213+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880038c0 con 0x7f27940b6100 2023-07-06T20:15:28.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.213+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279415a730 con 0x7f27940b6100 2023-07-06T20:15:28.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.213+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279415aa10 con 0x7f27940b6100 2023-07-06T20:15:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.214+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27941580e0 con 0x7f27940b6100 2023-07-06T20:15:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.215+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2788003a20 con 0x7f27940b6100 2023-07-06T20:15:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.216+0000 7f2790ff9700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f278006ace0 0x7f278006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.216+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2788088850 con 0x7f27940b6100 2023-07-06T20:15:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.216+0000 7f2793fff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f278006ace0 0x7f278006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.217+0000 7f2793fff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f278006ace0 0x7f278006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27840090b0 tx=0x7f278401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.220+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f27941580e0 con 0x7f27940b6100 2023-07-06T20:15:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.428+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27941580e0 con 0x7f27940b6100 2023-07-06T20:15:28.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.428+0000 7f2790ff9700 1 -- 172.21.15.27:0/3160625258 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2788017020 con 0x7f27940b6100 2023-07-06T20:15:28.430 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:15:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f278006ace0 msgr2=0x7f278006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f278006ace0 0x7f278006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27840090b0 tx=0x7f278401c040 comp rx=0 tx=0).stop 2023-07-06T20:15:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 msgr2=0x7f27940b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 0x7f27940b3150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f278800c520 tx=0x7f278800c830 comp rx=0 tx=0).stop 2023-07-06T20:15:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 shutdown_connections 2023-07-06T20:15:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f278006ace0 0x7f278006d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f279415fca0 0x7f2794153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f27940b5c90 0x7f27940cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.430+0000 7f279a9ff700 1 --2- 172.21.15.27:0/3160625258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27940b6100 0x7f27940b3150 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.431+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 >> 172.21.15.27:0/3160625258 conn(0x7f279406c150 msgr2=0x7f27940d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:28.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.431+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 shutdown_connections 2023-07-06T20:15:28.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.431+0000 7f279a9ff700 1 -- 172.21.15.27:0/3160625258 wait complete. 2023-07-06T20:15:28.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:15:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.656+0000 7fe1ad7ff700 1 Processor -- start 2023-07-06T20:15:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.656+0000 7fe1ad7ff700 1 -- start start 2023-07-06T20:15:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe1ad7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a815fca0 0x7fe1a80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe1ad7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe1a80b5c90 0x7fe1a80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe1ad7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a80cf780 con 0x7fe1a80b6100 2023-07-06T20:15:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe1ad7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a80cd490 con 0x7fe1a80b5c90 2023-07-06T20:15:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe1ad7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a80cd770 con 0x7fe1a815fca0 2023-07-06T20:15:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe19ffff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe1a80b5c90 0x7fe1a80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe19ffff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe1a80b5c90 0x7fe1a80d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51676/0 (socket says 172.21.15.27:51676) 2023-07-06T20:15:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.657+0000 7fe19ffff700 1 -- 172.21.15.27:0/2746096660 learned_addr learned my addr 172.21.15.27:0/2746096660 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe1a6ffd700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a815fca0 0x7fe1a80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe19ffff700 1 -- 172.21.15.27:0/2746096660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a815fca0 msgr2=0x7fe1a80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe19ffff700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a815fca0 0x7fe1a80d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe19ffff700 1 -- 172.21.15.27:0/2746096660 >> v1:172.21.15.153:6789/0 conn(0x7fe1a80b6100 legacy=0x7fe1a80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe19ffff700 1 -- 172.21.15.27:0/2746096660 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a80cda50 con 0x7fe1a80b5c90 2023-07-06T20:15:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe1a6ffd700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a815fca0 0x7fe1a80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:28.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe19ffff700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a80d2d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1900098f0 tx=0x7fe190009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d06533c608a02d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe1a67fc700 1 -- 172.21.15.27:0/2746096660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000b8a0 con 0x7fe1a80b5c90 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe1a67fc700 1 -- 172.21.15.27:0/2746096660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19000ba00 con 0x7fe1a80b5c90 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.658+0000 7fe1a67fc700 1 -- 172.21.15.27:0/2746096660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000bd90 con 0x7fe1a80b5c90 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/2746096660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 msgr2=0x7fe1a80d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a80d2d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1900098f0 tx=0x7fe190009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/2746096660 shutdown_connections 2023-07-06T20:15:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a815fca0 0x7fe1a80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/2746096660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a80d2d40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/2746096660 >> 172.21.15.27:0/2746096660 conn(0x7fe1a806c150 msgr2=0x7fe1a806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/2746096660 shutdown_connections 2023-07-06T20:15:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.659+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/2746096660 wait complete. 2023-07-06T20:15:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.661+0000 7fe1ad7ff700 1 Processor -- start 2023-07-06T20:15:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.661+0000 7fe1ad7ff700 1 -- start start 2023-07-06T20:15:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1ad7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1ad7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a80b6100 0x7fe1a80b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1ad7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1a815fca0 0x7fe1a80d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1ad7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a814c820 con 0x7fe1a80b5c90 2023-07-06T20:15:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1ad7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a815c4a0 con 0x7fe1a815fca0 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1ad7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a815c740 con 0x7fe1a80b6100 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1a6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1a6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a814c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51680/0 (socket says 172.21.15.27:51680) 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1a6ffd700 1 -- 172.21.15.27:0/1324317153 learned_addr learned my addr 172.21.15.27:0/1324317153 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1a77fe700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1a815fca0 0x7fe1a80d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe19ffff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a80b6100 0x7fe1a80b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1a6ffd700 1 -- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a80b6100 msgr2=0x7fe1a80b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.662+0000 7fe1a6ffd700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a80b6100 0x7fe1a80b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe1a6ffd700 1 -- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1a815fca0 msgr2=0x7fe1a80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe1a6ffd700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1a815fca0 0x7fe1a80d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe1a6ffd700 1 -- 172.21.15.27:0/1324317153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190009550 con 0x7fe1a80b5c90 2023-07-06T20:15:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe1a6ffd700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a814c2e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe19800e810 tx=0x7fe19800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800c020 con 0x7fe1a80b5c90 2023-07-06T20:15:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe198004250 con 0x7fe1a80b5c90 2023-07-06T20:15:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a815ca20 con 0x7fe1a80b5c90 2023-07-06T20:15:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1980036a0 con 0x7fe1a80b5c90 2023-07-06T20:15:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.663+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a815cfe0 con 0x7fe1a80b5c90 2023-07-06T20:15:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.664+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a80008d0 con 0x7fe1a80b5c90 2023-07-06T20:15:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.665+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe198003800 con 0x7fe1a80b5c90 2023-07-06T20:15:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.666+0000 7fe19f7fe700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe18806ace0 0x7fe18806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.666+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe198088fe0 con 0x7fe1a80b5c90 2023-07-06T20:15:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.667+0000 7fe19ffff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe18806ace0 0x7fe18806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.668+0000 7fe19ffff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe18806ace0 0x7fe18806d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe190009c40 tx=0x7fe190008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:28.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.671+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe198054030 con 0x7fe1a80b5c90 2023-07-06T20:15:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.882+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe1a80d06b0 con 0x7fe1a80b5c90 2023-07-06T20:15:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.882+0000 7fe19f7fe700 1 -- 172.21.15.27:0/1324317153 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe198057650 con 0x7fe1a80b5c90 2023-07-06T20:15:28.885 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:15:28.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.884+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe18806ace0 msgr2=0x7fe18806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.884+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe18806ace0 0x7fe18806d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe190009c40 tx=0x7fe190008030 comp rx=0 tx=0).stop 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.884+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 msgr2=0x7fe1a814c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.884+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a814c2e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe19800e810 tx=0x7fe19800ebd0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 shutdown_connections 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe18806ace0 0x7fe18806d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1a815fca0 0x7fe1a80d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a80b5c90 0x7fe1a814c2e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 --2- 172.21.15.27:0/1324317153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1a80b6100 0x7fe1a80b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 >> 172.21.15.27:0/1324317153 conn(0x7fe1a806c150 msgr2=0x7fe1a80b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 shutdown_connections 2023-07-06T20:15:28.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:28.885+0000 7fe1ad7ff700 1 -- 172.21.15.27:0/1324317153 wait complete. 2023-07-06T20:15:28.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:15:29.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.109+0000 7f97ac3ff700 1 Processor -- start 2023-07-06T20:15:29.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.110+0000 7f97ac3ff700 1 -- start start 2023-07-06T20:15:29.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.111+0000 7f97ac3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 0x7f97a40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.111+0000 7f97ac3ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97a40b6100 0x7f97a40cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.111+0000 7f97ac3ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40cf780 con 0x7f97a40b5c90 2023-07-06T20:15:29.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.111+0000 7f97ac3ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a40cd490 con 0x7f97a40b6100 2023-07-06T20:15:29.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.111+0000 7f97ac3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40cd770 con 0x7f97a415fca0 2023-07-06T20:15:29.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.111+0000 7f97aa19b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 0x7f97a40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97a999a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f97a40b5c90 0x7f97a40d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:50264/0 (socket says 172.21.15.27:50264) 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa19b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 0x7f97a40d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:50950/0 (socket says 172.21.15.27:50950) 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa99c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97a40b6100 0x7f97a40cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa19b700 1 -- 172.21.15.27:0/432603277 learned_addr learned my addr 172.21.15.27:0/432603277 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97a9199700 1 -- 172.21.15.27:0/432603277 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786819604 0 0) 0x7f97a40cf780 con 0x7f97a40b5c90 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97a9199700 1 -- 172.21.15.27:0/432603277 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97900035b0 con 0x7f97a40b5c90 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa99c700 1 -- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 msgr2=0x7f97a40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:29.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa99c700 1 --2- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 0x7f97a40d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa99c700 1 -- 172.21.15.27:0/432603277 >> v1:172.21.15.153:6789/0 conn(0x7f97a40b5c90 legacy=0x7f97a40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa99c700 1 -- 172.21.15.27:0/432603277 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40cda50 con 0x7f97a40b6100 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa19b700 1 --2- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 0x7f97a40d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.112+0000 7f97aa99c700 1 --2- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a40b6100 0x7f97a40cfc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f979c009d90 tx=0x7f979c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf1d6ec1ee48bcd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97a9199700 1 -- 172.21.15.27:0/432603277 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00e070 con 0x7f97a40b6100 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97a9199700 1 -- 172.21.15.27:0/432603277 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00bb20 con 0x7f97a40b6100 2023-07-06T20:15:29.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97a9199700 1 -- 172.21.15.27:0/432603277 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c012430 con 0x7f97a40b6100 2023-07-06T20:15:29.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 -- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a40b6100 msgr2=0x7f97a40cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a40b6100 0x7f97a40cfc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f979c009d90 tx=0x7f979c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:29.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 -- 172.21.15.27:0/432603277 shutdown_connections 2023-07-06T20:15:29.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a415fca0 0x7f97a40d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/432603277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a40b6100 0x7f97a40cfc10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 -- 172.21.15.27:0/432603277 >> 172.21.15.27:0/432603277 conn(0x7f97a406c150 msgr2=0x7f97a406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 -- 172.21.15.27:0/432603277 shutdown_connections 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.113+0000 7f97ac3ff700 1 -- 172.21.15.27:0/432603277 wait complete. 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.115+0000 7f97ac3ff700 1 Processor -- start 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.115+0000 7f97ac3ff700 1 -- start start 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.116+0000 7f97ac3ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 0x7f97a4149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.116+0000 7f97ac3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a40b6100 0x7f97a40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.116+0000 7f97ac3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a415fca0 0x7f97a40b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.116+0000 7f97ac3ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4147320 con 0x7f97a40b5c90 2023-07-06T20:15:29.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.116+0000 7f97ac3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4149c40 con 0x7f97a415fca0 2023-07-06T20:15:29.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.116+0000 7f97ac3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40cc020 con 0x7f97a40b6100 2023-07-06T20:15:29.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 0x7f97a4149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 0x7f97a4149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51686/0 (socket says 172.21.15.27:51686) 2023-07-06T20:15:29.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 -- 172.21.15.27:0/2552399559 learned_addr learned my addr 172.21.15.27:0/2552399559 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:29.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97a999a700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a40b6100 0x7f97a40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa99c700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a415fca0 0x7f97a40b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 -- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a40b6100 msgr2=0x7f97a40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a40b6100 0x7f97a40b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 -- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a415fca0 msgr2=0x7f97a40b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a415fca0 0x7f97a40b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 -- 172.21.15.27:0/2552399559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009550 con 0x7f97a40b5c90 2023-07-06T20:15:29.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.117+0000 7f97aa19b700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 0x7f97a4149700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e9b0 tx=0x7f97a000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:29.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.118+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000ee50 con 0x7f97a40b5c90 2023-07-06T20:15:29.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.118+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000c850 con 0x7f97a40b5c90 2023-07-06T20:15:29.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.118+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000cbc0 con 0x7f97a40b5c90 2023-07-06T20:15:29.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.118+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c009a70 con 0x7f97a40b5c90 2023-07-06T20:15:29.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.118+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a4146c60 con 0x7f97a40b5c90 2023-07-06T20:15:29.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.119+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4147460 con 0x7f97a40b5c90 2023-07-06T20:15:29.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.120+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97a000cd20 con 0x7f97a40b5c90 2023-07-06T20:15:29.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.121+0000 7f979affd700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f978806ace0 0x7f978806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.121+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97a0089440 con 0x7f97a40b5c90 2023-07-06T20:15:29.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.121+0000 7f97a999a700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f978806ace0 0x7f978806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.122+0000 7f97a999a700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f978806ace0 0x7f978806d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97a40b3ad0 tx=0x7f9794007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:29.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.125+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97a4147460 con 0x7f97a40b5c90 2023-07-06T20:15:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.331+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97a4147460 con 0x7f97a40b5c90 2023-07-06T20:15:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.331+0000 7f979affd700 1 -- 172.21.15.27:0/2552399559 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97a4147460 con 0x7f97a40b5c90 2023-07-06T20:15:29.333 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:15:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f978806ace0 msgr2=0x7f978806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f978806ace0 0x7f978806d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97a40b3ad0 tx=0x7f9794007dc0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 msgr2=0x7f97a4149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 0x7f97a4149700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e9b0 tx=0x7f97a000ecc0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 shutdown_connections 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f978806ace0 0x7f978806d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97a415fca0 0x7f97a40b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a40b5c90 0x7f97a4149700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 --2- 172.21.15.27:0/2552399559 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97a40b6100 0x7f97a40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 >> 172.21.15.27:0/2552399559 conn(0x7f97a406c150 msgr2=0x7f97a406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 shutdown_connections 2023-07-06T20:15:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.334+0000 7f97ac3ff700 1 -- 172.21.15.27:0/2552399559 wait complete. 2023-07-06T20:15:29.354 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:15:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.559+0000 7f94e47ff700 1 Processor -- start 2023-07-06T20:15:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.560+0000 7f94e47ff700 1 -- start start 2023-07-06T20:15:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e47ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94dc0b6110 0x7f94dc0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e47ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0cf780 con 0x7f94dc15fca0 2023-07-06T20:15:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e47ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0cd490 con 0x7f94dc0b5c90 2023-07-06T20:15:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e47ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0cd770 con 0x7f94dc0b6110 2023-07-06T20:15:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e259b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f94dc15fca0 0x7f94dc0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49336/0 (socket says 172.21.15.27:49336) 2023-07-06T20:15:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e259b700 1 -- 172.21.15.27:0/348008646 learned_addr learned my addr 172.21.15.27:0/348008646 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:29.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e2d9c700 1 --2- 172.21.15.27:0/348008646 >> v2:172.21.15.27:3300/0 conn(0x7f94dc0b6110 0x7f94dc0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.561+0000 7f94e1599700 1 -- 172.21.15.27:0/348008646 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388890833 0 0) 0x7f94dc0cf780 con 0x7f94dc15fca0 2023-07-06T20:15:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e1599700 1 -- 172.21.15.27:0/348008646 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c80035b0 con 0x7f94dc15fca0 2023-07-06T20:15:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e2d9c700 1 -- 172.21.15.27:0/348008646 >> v1:172.21.15.153:6789/0 conn(0x7f94dc0b5c90 legacy=0x7f94dc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e2d9c700 1 -- 172.21.15.27:0/348008646 >> v1:172.21.15.27:6789/0 conn(0x7f94dc15fca0 legacy=0x7f94dc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e2d9c700 1 -- 172.21.15.27:0/348008646 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0cda50 con 0x7f94dc0b6110 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e2d9c700 1 --2- 172.21.15.27:0/348008646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b6110 0x7f94dc0cfc10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05449a25a0ed6a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e1599700 1 -- 172.21.15.27:0/348008646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400e070 con 0x7f94dc0b6110 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e1599700 1 -- 172.21.15.27:0/348008646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400bb20 con 0x7f94dc0b6110 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.562+0000 7f94e1599700 1 -- 172.21.15.27:0/348008646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4012430 con 0x7f94dc0b6110 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 -- 172.21.15.27:0/348008646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b6110 msgr2=0x7f94dc0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 --2- 172.21.15.27:0/348008646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b6110 0x7f94dc0cfc10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 -- 172.21.15.27:0/348008646 shutdown_connections 2023-07-06T20:15:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 --2- 172.21.15.27:0/348008646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b6110 0x7f94dc0cfc10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 -- 172.21.15.27:0/348008646 >> 172.21.15.27:0/348008646 conn(0x7f94dc06c150 msgr2=0x7f94dc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 -- 172.21.15.27:0/348008646 shutdown_connections 2023-07-06T20:15:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.563+0000 7f94e47ff700 1 -- 172.21.15.27:0/348008646 wait complete. 2023-07-06T20:15:29.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.565+0000 7f94e47ff700 1 Processor -- start 2023-07-06T20:15:29.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.566+0000 7f94e47ff700 1 -- start start 2023-07-06T20:15:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e47ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 0x7f94dc1e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e47ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc0b6110 0x7f94dc0c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e47ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94dc15fca0 0x7f94dc0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e47ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1e78f0 con 0x7f94dc0b5c90 2023-07-06T20:15:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e47ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0ca990 con 0x7f94dc0b6110 2023-07-06T20:15:29.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e47ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0c7940 con 0x7f94dc15fca0 2023-07-06T20:15:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e259b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 0x7f94dc1e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e259b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 0x7f94dc1e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51714/0 (socket says 172.21.15.27:51714) 2023-07-06T20:15:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e259b700 1 -- 172.21.15.27:0/712711046 learned_addr learned my addr 172.21.15.27:0/712711046 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e1d9a700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc0b6110 0x7f94dc0c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.567+0000 7f94e2d9c700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94dc15fca0 0x7f94dc0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e259b700 1 -- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94dc15fca0 msgr2=0x7f94dc0ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e259b700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94dc15fca0 0x7f94dc0ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e259b700 1 -- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc0b6110 msgr2=0x7f94dc0c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e259b700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc0b6110 0x7f94dc0c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e259b700 1 -- 172.21.15.27:0/712711046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4009a70 con 0x7f94dc0b5c90 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e2d9c700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94dc15fca0 0x7f94dc0ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e1d9a700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc0b6110 0x7f94dc0c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e259b700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 0x7f94dc1e7d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f94d8001eb0 tx=0x7f94d800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8005d70 con 0x7f94dc0b5c90 2023-07-06T20:15:29.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d80070d0 con 0x7f94dc0b5c90 2023-07-06T20:15:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc1e6f50 con 0x7f94dc0b5c90 2023-07-06T20:15:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8007460 con 0x7f94dc0b5c90 2023-07-06T20:15:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.568+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc1e7510 con 0x7f94dc0b5c90 2023-07-06T20:15:29.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.569+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc1e7a30 con 0x7f94dc0b5c90 2023-07-06T20:15:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.570+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94d80075c0 con 0x7f94dc0b5c90 2023-07-06T20:15:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.571+0000 7f94d2ffd700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.571+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94d8088a10 con 0x7f94dc0b5c90 2023-07-06T20:15:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.572+0000 7f94e1d9a700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:29.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.572+0000 7f94e1d9a700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94c006ace0 0x7f94c006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94cc000f60 tx=0x7f94cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:29.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.576+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94dc1e7a30 con 0x7f94dc0b5c90 2023-07-06T20:15:29.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.814+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94dc1e7a30 con 0x7f94dc0b5c90 2023-07-06T20:15:29.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.814+0000 7f94d2ffd700 1 -- 172.21.15.27:0/712711046 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94dc1e7a30 con 0x7f94dc0b5c90 2023-07-06T20:15:29.817 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:15:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94c006ace0 msgr2=0x7f94c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94c006ace0 0x7f94c006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94cc000f60 tx=0x7f94cc009040 comp rx=0 tx=0).stop 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 msgr2=0x7f94dc1e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 0x7f94dc1e7d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f94d8001eb0 tx=0x7f94d800c860 comp rx=0 tx=0).stop 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 shutdown_connections 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94dc15fca0 0x7f94dc0ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc0b5c90 0x7f94dc1e7d10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 --2- 172.21.15.27:0/712711046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94dc0b6110 0x7f94dc0c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 >> 172.21.15.27:0/712711046 conn(0x7f94dc06c150 msgr2=0x7f94dc0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 shutdown_connections 2023-07-06T20:15:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:29.817+0000 7f94e47ff700 1 -- 172.21.15.27:0/712711046 wait complete. 2023-07-06T20:15:29.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:15:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.040+0000 7f513a3ff700 1 Processor -- start 2023-07-06T20:15:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.040+0000 7f513a3ff700 1 -- start start 2023-07-06T20:15:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f513a3ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f513415fca0 0x7f51340d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f513a3ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340cf780 con 0x7f51340b5c90 2023-07-06T20:15:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f513a3ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340cd490 con 0x7f51340b6110 2023-07-06T20:15:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f513a3ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51340cd770 con 0x7f513415fca0 2023-07-06T20:15:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f5133fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f513415fca0 0x7f51340d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f51337fe700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f51340b5c90 0x7f51340d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49344/0 (socket says 172.21.15.27:49344) 2023-07-06T20:15:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f51337fe700 1 -- 172.21.15.27:0/1876303988 learned_addr learned my addr 172.21.15.27:0/1876303988 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f5133fff700 1 -- 172.21.15.27:0/1876303988 >> v1:172.21.15.153:6789/0 conn(0x7f51340b6110 legacy=0x7f51340cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f5133fff700 1 -- 172.21.15.27:0/1876303988 >> v1:172.21.15.27:6789/0 conn(0x7f51340b5c90 legacy=0x7f51340d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f5133fff700 1 -- 172.21.15.27:0/1876303988 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340cda50 con 0x7f513415fca0 2023-07-06T20:15:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.042+0000 7f5132ffd700 1 -- 172.21.15.27:0/1876303988 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956394242 0 0) 0x7f51340cf780 con 0x7f51340b5c90 2023-07-06T20:15:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f5133fff700 1 --2- 172.21.15.27:0/1876303988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f513415fca0 0x7f51340d1420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5124009840 tx=0x7f5124009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20fd69ef38e848 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f5132ffd700 1 -- 172.21.15.27:0/1876303988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011830 con 0x7f513415fca0 2023-07-06T20:15:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f5132ffd700 1 -- 172.21.15.27:0/1876303988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124011990 con 0x7f513415fca0 2023-07-06T20:15:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f5132ffd700 1 -- 172.21.15.27:0/1876303988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011d20 con 0x7f513415fca0 2023-07-06T20:15:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f513a3ff700 1 -- 172.21.15.27:0/1876303988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f513415fca0 msgr2=0x7f51340d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.043+0000 7f513a3ff700 1 --2- 172.21.15.27:0/1876303988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f513415fca0 0x7f51340d1420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5124009840 tx=0x7f5124009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.044+0000 7f513a3ff700 1 -- 172.21.15.27:0/1876303988 shutdown_connections 2023-07-06T20:15:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.044+0000 7f513a3ff700 1 --2- 172.21.15.27:0/1876303988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f513415fca0 0x7f51340d1420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.044+0000 7f513a3ff700 1 -- 172.21.15.27:0/1876303988 >> 172.21.15.27:0/1876303988 conn(0x7f513406c150 msgr2=0x7f513406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.044+0000 7f513a3ff700 1 -- 172.21.15.27:0/1876303988 shutdown_connections 2023-07-06T20:15:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.044+0000 7f513a3ff700 1 -- 172.21.15.27:0/1876303988 wait complete. 2023-07-06T20:15:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.045+0000 7f513a3ff700 1 Processor -- start 2023-07-06T20:15:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.046+0000 7f513a3ff700 1 -- start start 2023-07-06T20:15:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513a3ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 0x7f5134157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513a3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51340b6110 0x7f51340d0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513a3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f513415fca0 0x7f51340d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513a3ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c6e20 con 0x7f51340b5c90 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513a3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340ccd40 con 0x7f513415fca0 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513a3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340cc780 con 0x7f51340b6110 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f5133fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 0x7f5134157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f5133fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 0x7f5134157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51746/0 (socket says 172.21.15.27:51746) 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f5133fff700 1 -- 172.21.15.27:0/3410457141 learned_addr learned my addr 172.21.15.27:0/3410457141 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f51337fe700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51340b6110 0x7f51340d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.047+0000 7f513899c700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f513415fca0 0x7f51340d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5133fff700 1 -- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51340b6110 msgr2=0x7f51340d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5133fff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51340b6110 0x7f51340d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5133fff700 1 -- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f513415fca0 msgr2=0x7f51340d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5133fff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f513415fca0 0x7f51340d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5133fff700 1 -- 172.21.15.27:0/3410457141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240094a0 con 0x7f51340b5c90 2023-07-06T20:15:30.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5133fff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 0x7f5134157320 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5124014040 tx=0x7f5124004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:30.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512401a7e0 con 0x7f51340b5c90 2023-07-06T20:15:30.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512401a940 con 0x7f51340b5c90 2023-07-06T20:15:30.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512401acb0 con 0x7f51340b5c90 2023-07-06T20:15:30.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51340cca60 con 0x7f51340b5c90 2023-07-06T20:15:30.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.048+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5134157860 con 0x7f51340b5c90 2023-07-06T20:15:30.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.049+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51340008d0 con 0x7f51340b5c90 2023-07-06T20:15:30.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.050+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5124022070 con 0x7f51340b5c90 2023-07-06T20:15:30.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.051+0000 7f5130ff9700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f512006ad10 0x7f512006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.051+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f512408a2c0 con 0x7f51340b5c90 2023-07-06T20:15:30.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.052+0000 7f51337fe700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f512006ad10 0x7f512006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.052+0000 7f51337fe700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f512006ad10 0x7f512006d1c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5128007b10 tx=0x7f5128007a60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:30.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.056+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51240555d0 con 0x7f51340b5c90 2023-07-06T20:15:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.281+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5134159d70 con 0x7f51340b5c90 2023-07-06T20:15:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.281+0000 7f5130ff9700 1 -- 172.21.15.27:0/3410457141 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5124058bf0 con 0x7f51340b5c90 2023-07-06T20:15:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.283+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f512006ad10 msgr2=0x7f512006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.283+0000 7f513a3ff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f512006ad10 0x7f512006d1c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5128007b10 tx=0x7f5128007a60 comp rx=0 tx=0).stop 2023-07-06T20:15:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.283+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 msgr2=0x7f5134157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.283+0000 7f513a3ff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 0x7f5134157320 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5124014040 tx=0x7f5124004750 comp rx=0 tx=0).stop 2023-07-06T20:15:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 shutdown_connections 2023-07-06T20:15:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f512006ad10 0x7f512006d1c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f513415fca0 0x7f51340d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51340b5c90 0x7f5134157320 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 --2- 172.21.15.27:0/3410457141 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51340b6110 0x7f51340d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 >> 172.21.15.27:0/3410457141 conn(0x7f513406c150 msgr2=0x7f51340b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 shutdown_connections 2023-07-06T20:15:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.284+0000 7f513a3ff700 1 -- 172.21.15.27:0/3410457141 wait complete. 2023-07-06T20:15:30.290 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-07-06T20:15:30.303 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T20:15:30.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:30.304 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-06T20:15:30.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:15:30.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:30.393 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T20:15:30.440 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:15:30.440 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:30.492 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:15:30.493 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:30.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:30.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.743+0000 7f952f9ff700 1 Processor -- start 2023-07-06T20:15:30.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.743+0000 7f952f9ff700 1 -- start start 2023-07-06T20:15:30.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.744+0000 7f952f9ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95280b5c90 0x7f95280d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.744+0000 7f952f9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.744+0000 7f952f9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95280cf780 con 0x7f952815fca0 2023-07-06T20:15:30.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.744+0000 7f952f9ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95280cd490 con 0x7f95280b5c90 2023-07-06T20:15:30.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.744+0000 7f952f9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280cd770 con 0x7f95280b6110 2023-07-06T20:15:30.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.744+0000 7f952cf9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95280b5c90 0x7f95280d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95280b5c90 0x7f95280d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51748/0 (socket says 172.21.15.27:51748) 2023-07-06T20:15:30.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 -- 172.21.15.27:0/1927529903 learned_addr learned my addr 172.21.15.27:0/1927529903 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:30.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952df9c700 1 --2- 172.21.15.27:0/1927529903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 -- 172.21.15.27:0/1927529903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 msgr2=0x7f95280d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:30.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 --2- 172.21.15.27:0/1927529903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 -- 172.21.15.27:0/1927529903 >> v1:172.21.15.153:6789/0 conn(0x7f952815fca0 legacy=0x7f95280d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 -- 172.21.15.27:0/1927529903 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280cda50 con 0x7f95280b5c90 2023-07-06T20:15:30.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f952cf9a700 1 --2- 172.21.15.27:0/1927529903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95280b5c90 0x7f95280d2d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95180098f0 tx=0x7f9518009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d22cfad3c7e316 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:30.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.745+0000 7f951ffff700 1 -- 172.21.15.27:0/1927529903 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518011830 con 0x7f95280b5c90 2023-07-06T20:15:30.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f951ffff700 1 -- 172.21.15.27:0/1927529903 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9518011990 con 0x7f95280b5c90 2023-07-06T20:15:30.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f951ffff700 1 -- 172.21.15.27:0/1927529903 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518011d20 con 0x7f95280b5c90 2023-07-06T20:15:30.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 -- 172.21.15.27:0/1927529903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95280b5c90 msgr2=0x7f95280d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 --2- 172.21.15.27:0/1927529903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95280b5c90 0x7f95280d2d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95180098f0 tx=0x7f9518009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 -- 172.21.15.27:0/1927529903 shutdown_connections 2023-07-06T20:15:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 --2- 172.21.15.27:0/1927529903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 --2- 172.21.15.27:0/1927529903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95280b5c90 0x7f95280d2d40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 -- 172.21.15.27:0/1927529903 >> 172.21.15.27:0/1927529903 conn(0x7f952806c150 msgr2=0x7f952806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 -- 172.21.15.27:0/1927529903 shutdown_connections 2023-07-06T20:15:30.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.746+0000 7f952f9ff700 1 -- 172.21.15.27:0/1927529903 wait complete. 2023-07-06T20:15:30.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.748+0000 7f952f9ff700 1 Processor -- start 2023-07-06T20:15:30.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.748+0000 7f952f9ff700 1 -- start start 2023-07-06T20:15:30.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952f9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95280b5c90 0x7f9528150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952f9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952f9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 0x7f95280c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952f9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528151420 con 0x7f952815fca0 2023-07-06T20:15:30.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952f9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280c8dc0 con 0x7f95280b5c90 2023-07-06T20:15:30.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952f9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95280c90a0 con 0x7f95280b6110 2023-07-06T20:15:30.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.749+0000 7f952df9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 0x7f95280c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952d79b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95280b5c90 0x7f9528150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952cf9a700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 0x7f95280c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51750/0 (socket says 172.21.15.27:51750) 2023-07-06T20:15:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 -- 172.21.15.27:0/134854059 learned_addr learned my addr 172.21.15.27:0/134854059 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 -- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 msgr2=0x7f95280c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 -- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95280b5c90 msgr2=0x7f9528150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95280b5c90 0x7f9528150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 -- 172.21.15.27:0/134854059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518009550 con 0x7f952815fca0 2023-07-06T20:15:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f952df9c700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 0x7f95280c8480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f952000efc0 tx=0x7f952000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:30.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520009ea0 con 0x7f952815fca0 2023-07-06T20:15:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952000f040 con 0x7f952815fca0 2023-07-06T20:15:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.750+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000ced0 con 0x7f952815fca0 2023-07-06T20:15:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.751+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95280c9380 con 0x7f952815fca0 2023-07-06T20:15:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.751+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95280c89c0 con 0x7f952815fca0 2023-07-06T20:15:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.752+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95280008d0 con 0x7f952815fca0 2023-07-06T20:15:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.753+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9520014480 con 0x7f952815fca0 2023-07-06T20:15:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.754+0000 7f951dffb700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f950c06ace0 0x7f950c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.754+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9520089d70 con 0x7f952815fca0 2023-07-06T20:15:30.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.754+0000 7f952d79b700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f950c06ace0 0x7f950c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.755+0000 7f952d79b700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f950c06ace0 0x7f950c06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9524007fd0 tx=0x7f9524009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.758+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9520054d90 con 0x7f952815fca0 2023-07-06T20:15:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.968+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9528157990 con 0x7f952815fca0 2023-07-06T20:15:30.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.969+0000 7f951dffb700 1 -- 172.21.15.27:0/134854059 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f9520018020 con 0x7f952815fca0 2023-07-06T20:15:30.971 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:30.971 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:18.212485+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:30.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.971+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f950c06ace0 msgr2=0x7f950c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.971+0000 7f952f9ff700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f950c06ace0 0x7f950c06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9524007fd0 tx=0x7f9524009040 comp rx=0 tx=0).stop 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.971+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 msgr2=0x7f95280c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.971+0000 7f952f9ff700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 0x7f95280c8480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f952000efc0 tx=0x7f952000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 shutdown_connections 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f950c06ace0 0x7f950c06d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952815fca0 0x7f95280c8480 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95280b5c90 0x7f9528150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 --2- 172.21.15.27:0/134854059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95280b6110 0x7f95280c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:30.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 >> 172.21.15.27:0/134854059 conn(0x7f952806c150 msgr2=0x7f952815a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:30.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 shutdown_connections 2023-07-06T20:15:30.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:30.972+0000 7f952f9ff700 1 -- 172.21.15.27:0/134854059 wait complete. 2023-07-06T20:15:30.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:15:15.163204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:15:15.707733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:15:30.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:15:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.197+0000 7ff614dff700 1 Processor -- start 2023-07-06T20:15:31.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.197+0000 7ff614dff700 1 -- start start 2023-07-06T20:15:31.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff614dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6100b0440 0x7ff610143900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff614dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 0x7ff61015b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff614dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6101441f0 con 0x7ff61015df60 2023-07-06T20:15:31.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff614dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61015e3e0 con 0x7ff6100b0440 2023-07-06T20:15:31.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff614dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015a930 con 0x7ff61015b0e0 2023-07-06T20:15:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff607fff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff61015df60 0x7ff6100b0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49346/0 (socket says 172.21.15.27:49346) 2023-07-06T20:15:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.198+0000 7ff607fff700 1 -- 172.21.15.27:0/3953441740 learned_addr learned my addr 172.21.15.27:0/3953441740 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3953441740 >> v2:172.21.15.153:3300/0 conn(0x7ff6100b0440 0x7ff610143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60ed9d700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 0x7ff61015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011774771 0 0) 0x7ff6101441f0 con 0x7ff61015df60 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5fc0035b0 con 0x7ff61015df60 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1305992863 0 0) 0x7ff5fc0035b0 con 0x7ff61015df60 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 msgr2=0x7ff61015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 0x7ff61015b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 >> v2:172.21.15.153:3300/0 conn(0x7ff6100b0440 msgr2=0x7ff610143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 --2- 172.21.15.27:0/3953441740 >> v2:172.21.15.153:3300/0 conn(0x7ff6100b0440 0x7ff610143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61015e660 con 0x7ff61015df60 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7ff5f8001670 con 0x7ff61015df60 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 >> v1:172.21.15.27:6789/0 conn(0x7ff61015df60 legacy=0x7ff6100b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc003a00 0x7ff5fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 0x7ff5fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5fc004d90 0x7ff5fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015e3e0 con 0x7ff5fc004d90 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015a930 con 0x7ff5fc003a00 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.199+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc00ba10 con 0x7ff61015b0e0 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc003a00 0x7ff5fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff607fff700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 0x7ff5fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60ed9d700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5fc004d90 0x7ff5fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 -- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 msgr2=0x7ff5fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff61015b0e0 0x7ff5fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 -- 172.21.15.27:0/3953441740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5fc004d90 msgr2=0x7ff5fc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5fc004d90 0x7ff5fc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 -- 172.21.15.27:0/3953441740 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0035b0 con 0x7ff5fc003a00 2023-07-06T20:15:31.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.200+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc003a00 0x7ff5fc003e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff600009b60 tx=0x7ff5fc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc0378fe2023186 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:31.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000e6c0 con 0x7ff5fc003a00 2023-07-06T20:15:31.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000e820 con 0x7ff5fc003a00 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff60dd9b700 1 -- 172.21.15.27:0/3953441740 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000eb90 con 0x7ff5fc003a00 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff614dff700 1 -- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc003a00 msgr2=0x7ff5fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff614dff700 1 --2- 172.21.15.27:0/3953441740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5fc003a00 0x7ff5fc003e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff600009b60 tx=0x7ff5fc005880 comp rx=0 tx=0).stop 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff60ed9d700 1 -- 172.21.15.27:0/3953441740 reap_dead start 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff614dff700 1 -- 172.21.15.27:0/3953441740 shutdown_connections 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff614dff700 1 -- 172.21.15.27:0/3953441740 >> 172.21.15.27:0/3953441740 conn(0x7ff6100cb440 msgr2=0x7ff6100cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.201+0000 7ff614dff700 1 -- 172.21.15.27:0/3953441740 shutdown_connections 2023-07-06T20:15:31.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.202+0000 7ff614dff700 1 -- 172.21.15.27:0/3953441740 wait complete. 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.203+0000 7ff614dff700 1 Processor -- start 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.204+0000 7ff614dff700 1 -- start start 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.204+0000 7ff614dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5fc003a00 0x7ff6100cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff614dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6100b2a10 0x7ff6100c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff614dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 0x7ff6100accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff614dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100ad230 con 0x7ff6100ac880 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff614dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b3d90 con 0x7ff6100b2a10 2023-07-06T20:15:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff614dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b3180 con 0x7ff5fc003a00 2023-07-06T20:15:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 0x7ff6100accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 0x7ff6100accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51778/0 (socket says 172.21.15.27:51778) 2023-07-06T20:15:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 -- 172.21.15.27:0/3050144553 learned_addr learned my addr 172.21.15.27:0/3050144553 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5fc003a00 0x7ff6100cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff607fff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6100b2a10 0x7ff6100c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 -- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5fc003a00 msgr2=0x7ff6100cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5fc003a00 0x7ff6100cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 -- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6100b2a10 msgr2=0x7ff6100c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6100b2a10 0x7ff6100c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.205+0000 7ff60ed9d700 1 -- 172.21.15.27:0/3050144553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100b3460 con 0x7ff6100ac880 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.206+0000 7ff60ed9d700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 0x7ff6100accf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff610071c00 tx=0x7ff608011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.206+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800e040 con 0x7ff6100ac880 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.206+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800f040 con 0x7ff6100ac880 2023-07-06T20:15:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.206+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6100b3740 con 0x7ff6100ac880 2023-07-06T20:15:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.206+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800fc10 con 0x7ff6100ac880 2023-07-06T20:15:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.206+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6100b2e80 con 0x7ff6100ac880 2023-07-06T20:15:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.207+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61006fa50 con 0x7ff6100ac880 2023-07-06T20:15:31.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.208+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff60800fd70 con 0x7ff6100ac880 2023-07-06T20:15:31.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.209+0000 7ff606ffd700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff5f006ace0 0x7ff5f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.209+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff608089940 con 0x7ff6100ac880 2023-07-06T20:15:31.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.209+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff5f006ace0 0x7ff5f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.210+0000 7ff60e59c700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff5f006ace0 0x7ff5f006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff6000090b0 tx=0x7ff60001f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:31.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.214+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff6080583e0 con 0x7ff6100ac880 2023-07-06T20:15:31.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.420+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6100008d0 con 0x7ff6100ac880 2023-07-06T20:15:31.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.420+0000 7ff606ffd700 1 -- 172.21.15.27:0/3050144553 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff608057f70 con 0x7ff6100ac880 2023-07-06T20:15:31.423 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:15:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff5f006ace0 msgr2=0x7ff5f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff5f006ace0 0x7ff5f006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff6000090b0 tx=0x7ff60001f040 comp rx=0 tx=0).stop 2023-07-06T20:15:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 msgr2=0x7ff6100accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 0x7ff6100accf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff610071c00 tx=0x7ff608011ed0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 shutdown_connections 2023-07-06T20:15:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff5f006ace0 0x7ff5f006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6100ac880 0x7ff6100accf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff5fc003a00 0x7ff6100cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 --2- 172.21.15.27:0/3050144553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6100b2a10 0x7ff6100c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 >> 172.21.15.27:0/3050144553 conn(0x7ff6100cb440 msgr2=0x7ff61006c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 shutdown_connections 2023-07-06T20:15:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.423+0000 7ff614dff700 1 -- 172.21.15.27:0/3050144553 wait complete. 2023-07-06T20:15:31.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:15:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.647+0000 7fd310bff700 1 Processor -- start 2023-07-06T20:15:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.647+0000 7fd310bff700 1 -- start start 2023-07-06T20:15:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd310bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 0x7fd30c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd310bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd30c0c9010 0x7fd30c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd310bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c06cc40 con 0x7fd30c1565f0 2023-07-06T20:15:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd310bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0c9a60 con 0x7fd30c0c9010 2023-07-06T20:15:31.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd310bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c159b70 con 0x7fd30c06c330 2023-07-06T20:15:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd30ad9d700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd30c1565f0 0x7fd30c156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49350/0 (socket says 172.21.15.27:49350) 2023-07-06T20:15:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.648+0000 7fd30ad9d700 1 -- 172.21.15.27:0/2254406498 learned_addr learned my addr 172.21.15.27:0/2254406498 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30a59c700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 0x7fd30c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd309d9b700 1 --2- 172.21.15.27:0/2254406498 >> v2:172.21.15.153:3300/0 conn(0x7fd30c0c9010 0x7fd30c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854928539 0 0) 0x7fd30c06cc40 con 0x7fd30c1565f0 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f40035b0 con 0x7fd30c1565f0 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2763589947 0 0) 0x7fd2f40035b0 con 0x7fd30c1565f0 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 msgr2=0x7fd30c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 0x7fd30c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 >> v2:172.21.15.153:3300/0 conn(0x7fd30c0c9010 msgr2=0x7fd30c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 --2- 172.21.15.27:0/2254406498 >> v2:172.21.15.153:3300/0 conn(0x7fd30c0c9010 0x7fd30c0c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c159e50 con 0x7fd30c1565f0 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7fd3040019f0 con 0x7fd30c1565f0 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 >> v1:172.21.15.27:6789/0 conn(0x7fd30c1565f0 legacy=0x7fd30c156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f4003a00 0x7fd2f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 0x7fd2f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f4004d90 0x7fd2f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c9a60 con 0x7fd2f4003a00 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.649+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c159b70 con 0x7fd2f4004d90 2023-07-06T20:15:31.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f400ba90 con 0x7fd30c06c330 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f4003a00 0x7fd2f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd309d9b700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 0x7fd2f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30ad9d700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f4004d90 0x7fd2f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 -- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 msgr2=0x7fd2f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd30c06c330 0x7fd2f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 -- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f4004d90 msgr2=0x7fd2f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:31.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f4004d90 0x7fd2f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 -- 172.21.15.27:0/2254406498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40035b0 con 0x7fd2f4003a00 2023-07-06T20:15:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.650+0000 7fd30a59c700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f4003a00 0x7fd2f4003e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009b60 tx=0x7fd2f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede8d48bea14daa4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00e6c0 con 0x7fd2f4003a00 2023-07-06T20:15:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00e820 con 0x7fd2f4003a00 2023-07-06T20:15:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd30959a700 1 -- 172.21.15.27:0/2254406498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00eb90 con 0x7fd2f4003a00 2023-07-06T20:15:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd310bff700 1 -- 172.21.15.27:0/2254406498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f4003a00 msgr2=0x7fd2f4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd310bff700 1 --2- 172.21.15.27:0/2254406498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f4003a00 0x7fd2f4003e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009b60 tx=0x7fd2f4005880 comp rx=0 tx=0).stop 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd30ad9d700 1 -- 172.21.15.27:0/2254406498 reap_dead start 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd310bff700 1 -- 172.21.15.27:0/2254406498 shutdown_connections 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd310bff700 1 -- 172.21.15.27:0/2254406498 >> 172.21.15.27:0/2254406498 conn(0x7fd30c0cb440 msgr2=0x7fd30c0cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd310bff700 1 -- 172.21.15.27:0/2254406498 shutdown_connections 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.651+0000 7fd310bff700 1 -- 172.21.15.27:0/2254406498 wait complete. 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.654+0000 7fd310bff700 1 Processor -- start 2023-07-06T20:15:31.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.654+0000 7fd310bff700 1 -- start start 2023-07-06T20:15:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.655+0000 7fd310bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f4003a00 0x7fd30c0af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.655+0000 7fd310bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd30c1f6460 0x7fd30c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.655+0000 7fd310bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 0x7fd30c0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.655+0000 7fd310bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0ad230 con 0x7fd30c0ac880 2023-07-06T20:15:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.655+0000 7fd310bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b0e50 con 0x7fd30c1f6460 2023-07-06T20:15:31.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.655+0000 7fd310bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0afdb0 con 0x7fd2f4003a00 2023-07-06T20:15:31.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 0x7fd30c0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 0x7fd30c0accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51788/0 (socket says 172.21.15.27:51788) 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 -- 172.21.15.27:0/4226694477 learned_addr learned my addr 172.21.15.27:0/4226694477 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30a59c700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f4003a00 0x7fd30c0af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd309d9b700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd30c1f6460 0x7fd30c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 -- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f4003a00 msgr2=0x7fd30c0af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f4003a00 0x7fd30c0af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 -- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd30c1f6460 msgr2=0x7fd30c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd30c1f6460 0x7fd30c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 -- 172.21.15.27:0/4226694477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0b0090 con 0x7fd30c0ac880 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.656+0000 7fd30ad9d700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 0x7fd30c0accf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0b1550 tx=0x7fd304007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.657+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400eb10 con 0x7fd30c0ac880 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.657+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30400ec90 con 0x7fd30c0ac880 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.657+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304013420 con 0x7fd30c0ac880 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.657+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0b0370 con 0x7fd30c0ac880 2023-07-06T20:15:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.657+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0b0930 con 0x7fd30c0ac880 2023-07-06T20:15:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.658+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0008d0 con 0x7fd30c0ac880 2023-07-06T20:15:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.659+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd304013580 con 0x7fd30c0ac880 2023-07-06T20:15:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.660+0000 7fd2faffd700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd2ec06ace0 0x7fd2ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:31.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.660+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd304089920 con 0x7fd30c0ac880 2023-07-06T20:15:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.661+0000 7fd30a59c700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd2ec06ace0 0x7fd2ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:31.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.661+0000 7fd30a59c700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd2ec06ace0 0x7fd2ec06d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc000c00 tx=0x7fd2fc01f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:31.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.664+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd3040549d0 con 0x7fd30c0ac880 2023-07-06T20:15:31.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.872+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd30c0ae100 con 0x7fd30c0ac880 2023-07-06T20:15:31.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.873+0000 7fd2faffd700 1 -- 172.21.15.27:0/4226694477 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd304057ff0 con 0x7fd30c0ac880 2023-07-06T20:15:31.875 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:15:31.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd2ec06ace0 msgr2=0x7fd2ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd2ec06ace0 0x7fd2ec06d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc000c00 tx=0x7fd2fc01f040 comp rx=0 tx=0).stop 2023-07-06T20:15:31.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 msgr2=0x7fd30c0accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:31.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 0x7fd30c0accf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0b1550 tx=0x7fd304007c90 comp rx=0 tx=0).stop 2023-07-06T20:15:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 shutdown_connections 2023-07-06T20:15:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd2ec06ace0 0x7fd2ec06d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd30c0ac880 0x7fd30c0accf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f4003a00 0x7fd30c0af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 --2- 172.21.15.27:0/4226694477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd30c1f6460 0x7fd30c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 >> 172.21.15.27:0/4226694477 conn(0x7fd30c0cb440 msgr2=0x7fd30c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 shutdown_connections 2023-07-06T20:15:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:31.875+0000 7fd310bff700 1 -- 172.21.15.27:0/4226694477 wait complete. 2023-07-06T20:15:31.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:15:32.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.101+0000 7f131fdff700 1 Processor -- start 2023-07-06T20:15:32.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.101+0000 7f131fdff700 1 -- start start 2023-07-06T20:15:32.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131fdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f131815fca0 0x7f13180d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131fdff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f13180b5c90 0x7f13180d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131fdff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180cf780 con 0x7f13180b6110 2023-07-06T20:15:32.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131fdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180cd490 con 0x7f131815fca0 2023-07-06T20:15:32.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131fdff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180cd770 con 0x7f13180b5c90 2023-07-06T20:15:32.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131db9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f131815fca0 0x7f13180d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131db9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f131815fca0 0x7f13180d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42202/0 (socket says 172.21.15.27:42202) 2023-07-06T20:15:32.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.102+0000 7f131db9b700 1 -- 172.21.15.27:0/1919312709 learned_addr learned my addr 172.21.15.27:0/1919312709 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:32.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131d39a700 1 --2- 172.21.15.27:0/1919312709 >> v2:172.21.15.153:3300/0 conn(0x7f13180b5c90 0x7f13180d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131db9b700 1 -- 172.21.15.27:0/1919312709 >> v2:172.21.15.153:3300/0 conn(0x7f13180b5c90 msgr2=0x7f13180d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:32.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131db9b700 1 --2- 172.21.15.27:0/1919312709 >> v2:172.21.15.153:3300/0 conn(0x7f13180b5c90 0x7f13180d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131db9b700 1 -- 172.21.15.27:0/1919312709 >> v1:172.21.15.153:6789/0 conn(0x7f13180b6110 legacy=0x7f13180cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131db9b700 1 -- 172.21.15.27:0/1919312709 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180cda50 con 0x7f131815fca0 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131db9b700 1 --2- 172.21.15.27:0/1919312709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d1420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f131806b6f0 tx=0x7f1314009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9804ef5ff7edb93a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131cb99700 1 -- 172.21.15.27:0/1919312709 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131400b870 con 0x7f131815fca0 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131cb99700 1 -- 172.21.15.27:0/1919312709 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131400b9d0 con 0x7f131815fca0 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.103+0000 7f131cb99700 1 -- 172.21.15.27:0/1919312709 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131400bd60 con 0x7f131815fca0 2023-07-06T20:15:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 -- 172.21.15.27:0/1919312709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 msgr2=0x7f13180d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 --2- 172.21.15.27:0/1919312709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d1420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f131806b6f0 tx=0x7f1314009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 -- 172.21.15.27:0/1919312709 shutdown_connections 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 --2- 172.21.15.27:0/1919312709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d1420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 --2- 172.21.15.27:0/1919312709 >> v2:172.21.15.153:3300/0 conn(0x7f13180b5c90 0x7f13180d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 -- 172.21.15.27:0/1919312709 >> 172.21.15.27:0/1919312709 conn(0x7f131806c150 msgr2=0x7f131806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 -- 172.21.15.27:0/1919312709 shutdown_connections 2023-07-06T20:15:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.104+0000 7f131fdff700 1 -- 172.21.15.27:0/1919312709 wait complete. 2023-07-06T20:15:32.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.106+0000 7f131fdff700 1 Processor -- start 2023-07-06T20:15:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.106+0000 7f131fdff700 1 -- start start 2023-07-06T20:15:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131fdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13180b5c90 0x7f131814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131fdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13180b6110 0x7f13180c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131fdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131fdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180ca5c0 con 0x7f131815fca0 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131fdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180c9370 con 0x7f13180b6110 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131fdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13180c9650 con 0x7f13180b5c90 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131e39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131e39c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42208/0 (socket says 172.21.15.27:42208) 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131e39c700 1 -- 172.21.15.27:0/641007856 learned_addr learned my addr 172.21.15.27:0/641007856 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131db9b700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13180b5c90 0x7f131814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.107+0000 7f131d39a700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13180b6110 0x7f13180c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131e39c700 1 -- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13180b5c90 msgr2=0x7f131814c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131e39c700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13180b5c90 0x7f131814c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131e39c700 1 -- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13180b6110 msgr2=0x7f13180c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131e39c700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13180b6110 0x7f13180c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131e39c700 1 -- 172.21.15.27:0/641007856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1314009550 con 0x7f131815fca0 2023-07-06T20:15:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131d39a700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13180b6110 0x7f13180c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:32.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131e39c700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d2b60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f131000cc50 tx=0x7f131000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:32.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310003c00 con 0x7f131815fca0 2023-07-06T20:15:32.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1310003d60 con 0x7f131815fca0 2023-07-06T20:15:32.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131000ab60 con 0x7f131815fca0 2023-07-06T20:15:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13180ca810 con 0x7f131815fca0 2023-07-06T20:15:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.108+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318159190 con 0x7f131815fca0 2023-07-06T20:15:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.109+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13180cad30 con 0x7f131815fca0 2023-07-06T20:15:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.110+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f131000acc0 con 0x7f131815fca0 2023-07-06T20:15:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.111+0000 7f130e7fc700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12fc06ace0 0x7f12fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.111+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1310089430 con 0x7f131815fca0 2023-07-06T20:15:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.112+0000 7f131db9b700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12fc06ace0 0x7f12fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.113+0000 7f131db9b700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12fc06ace0 0x7f12fc06d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1314014040 tx=0x7f1314008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.116+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13180cad30 con 0x7f131815fca0 2023-07-06T20:15:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.321+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13180cad30 con 0x7f131815fca0 2023-07-06T20:15:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.322+0000 7f130e7fc700 1 -- 172.21.15.27:0/641007856 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13180cad30 con 0x7f131815fca0 2023-07-06T20:15:32.324 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:15:32.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12fc06ace0 msgr2=0x7f12fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12fc06ace0 0x7f12fc06d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1314014040 tx=0x7f1314008030 comp rx=0 tx=0).stop 2023-07-06T20:15:32.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 msgr2=0x7f13180d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d2b60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f131000cc50 tx=0x7f131000cf60 comp rx=0 tx=0).stop 2023-07-06T20:15:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 shutdown_connections 2023-07-06T20:15:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f12fc06ace0 0x7f12fc06d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131815fca0 0x7f13180d2b60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13180b5c90 0x7f131814c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 --2- 172.21.15.27:0/641007856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13180b6110 0x7f13180c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 >> 172.21.15.27:0/641007856 conn(0x7f131806c150 msgr2=0x7f13180b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:32.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 shutdown_connections 2023-07-06T20:15:32.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.324+0000 7f131fdff700 1 -- 172.21.15.27:0/641007856 wait complete. 2023-07-06T20:15:32.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:15:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.550+0000 7f9ad77ff700 1 Processor -- start 2023-07-06T20:15:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.550+0000 7f9ad77ff700 1 -- start start 2023-07-06T20:15:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.551+0000 7f9ad77ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9ad00b5c90 0x7f9ad00d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.551+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b6110 0x7f9ad00d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.551+0000 7f9ad77ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad00cf780 con 0x7f9ad015fca0 2023-07-06T20:15:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.551+0000 7f9ad77ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad00cd490 con 0x7f9ad00b5c90 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.551+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad00cd770 con 0x7f9ad00b6110 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad5d9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b6110 0x7f9ad00d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad5d9c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b6110 0x7f9ad00d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:45562/0 (socket says 172.21.15.27:45562) 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad5d9c700 1 -- 172.21.15.27:0/2671381833 learned_addr learned my addr 172.21.15.27:0/2671381833 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad4d9a700 1 --2- 172.21.15.27:0/2671381833 >> v2:172.21.15.153:3300/0 conn(0x7f9ad00b5c90 0x7f9ad00d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b6110 msgr2=0x7f9ad00d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad4d9a700 1 --2- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b6110 0x7f9ad00d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2671381833 >> v1:172.21.15.153:6789/0 conn(0x7f9ad015fca0 legacy=0x7f9ad00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.552+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2671381833 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00cda50 con 0x7f9ad00b5c90 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ad4d9a700 1 --2- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad00b5c90 0x7f9ad00d2d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009840 tx=0x7f9ac0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c8c4928c9fde242 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ac7fff700 1 -- 172.21.15.27:0/2671381833 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0011830 con 0x7f9ad00b5c90 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ac7fff700 1 -- 172.21.15.27:0/2671381833 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0011990 con 0x7f9ad00b5c90 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ac7fff700 1 -- 172.21.15.27:0/2671381833 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0011d20 con 0x7f9ad00b5c90 2023-07-06T20:15:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad00b5c90 msgr2=0x7f9ad00d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad00b5c90 0x7f9ad00d2d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009840 tx=0x7f9ac0009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2671381833 shutdown_connections 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.553+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b6110 0x7f9ad00d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.554+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2671381833 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad00b5c90 0x7f9ad00d2d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.554+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2671381833 >> 172.21.15.27:0/2671381833 conn(0x7f9ad006c150 msgr2=0x7f9ad006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.554+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2671381833 shutdown_connections 2023-07-06T20:15:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.554+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2671381833 wait complete. 2023-07-06T20:15:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.555+0000 7f9ad77ff700 1 Processor -- start 2023-07-06T20:15:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.555+0000 7f9ad77ff700 1 -- start start 2023-07-06T20:15:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.556+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b5c90 0x7f9ad00c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.556+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 0x7f9ad00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.556+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad015fca0 0x7f9ad015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.556+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad00c8f60 con 0x7f9ad00b6110 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.556+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01f3110 con 0x7f9ad015fca0 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.556+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad00c6fa0 con 0x7f9ad00b5c90 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 0x7f9ad00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 0x7f9ad00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42222/0 (socket says 172.21.15.27:42222) 2023-07-06T20:15:32.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2402417462 learned_addr learned my addr 172.21.15.27:0/2402417462 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad5d9c700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad015fca0 0x7f9ad015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad559b700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b5c90 0x7f9ad00c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b5c90 msgr2=0x7f9ad00c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b5c90 0x7f9ad00c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad015fca0 msgr2=0x7f9ad015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad015fca0 0x7f9ad015a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 -- 172.21.15.27:0/2402417462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00094a0 con 0x7f9ad00b6110 2023-07-06T20:15:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.557+0000 7f9ad4d9a700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 0x7f9ad00c8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009fd0 tx=0x7f9ac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.558+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00036a0 con 0x7f9ad00b6110 2023-07-06T20:15:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.558+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac001f070 con 0x7f9ad00b6110 2023-07-06T20:15:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.558+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0003a60 con 0x7f9ad00b6110 2023-07-06T20:15:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.558+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad00c7280 con 0x7f9ad00b6110 2023-07-06T20:15:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.558+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad00c7840 con 0x7f9ad00b6110 2023-07-06T20:15:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.559+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad01f3280 con 0x7f9ad00b6110 2023-07-06T20:15:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.560+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ac0003bc0 con 0x7f9ad00b6110 2023-07-06T20:15:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.561+0000 7f9ac5ffb700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9ab406ad10 0x7f9ab406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.561+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ac0021070 con 0x7f9ad00b6110 2023-07-06T20:15:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.561+0000 7f9ad559b700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9ab406ad10 0x7f9ab406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.562+0000 7f9ad559b700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9ab406ad10 0x7f9ab406d1c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9acc007fd0 tx=0x7f9acc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.565+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ac0055ce0 con 0x7f9ad00b6110 2023-07-06T20:15:32.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.768+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ad00008d0 con 0x7f9ad00b6110 2023-07-06T20:15:32.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.768+0000 7f9ac5ffb700 1 -- 172.21.15.27:0/2402417462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ac00150a0 con 0x7f9ad00b6110 2023-07-06T20:15:32.770 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:15:32.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9ab406ad10 msgr2=0x7f9ab406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9ab406ad10 0x7f9ab406d1c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9acc007fd0 tx=0x7f9acc009040 comp rx=0 tx=0).stop 2023-07-06T20:15:32.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 msgr2=0x7f9ad00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:32.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 0x7f9ad00c8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009fd0 tx=0x7f9ac0004750 comp rx=0 tx=0).stop 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 shutdown_connections 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f9ab406ad10 0x7f9ab406d1c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ad015fca0 0x7f9ad015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9ad00b5c90 0x7f9ad00c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 --2- 172.21.15.27:0/2402417462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ad00b6110 0x7f9ad00c8a20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 >> 172.21.15.27:0/2402417462 conn(0x7f9ad006c150 msgr2=0x7f9ad00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 shutdown_connections 2023-07-06T20:15:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.771+0000 7f9ad77ff700 1 -- 172.21.15.27:0/2402417462 wait complete. 2023-07-06T20:15:32.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:15:32.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.994+0000 7f1e681ff700 1 Processor -- start 2023-07-06T20:15:32.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.994+0000 7f1e681ff700 1 -- start start 2023-07-06T20:15:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.995+0000 7f1e681ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.995+0000 7f1e681ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1e600b5c90 0x7f1e600d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.995+0000 7f1e681ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e600cf780 con 0x7f1e600b6100 2023-07-06T20:15:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e681ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e600cd490 con 0x7f1e600b5c90 2023-07-06T20:15:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e681ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600cd770 con 0x7f1e6015fca0 2023-07-06T20:15:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6679c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1e600b6100 0x7f1e600cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60656/0 (socket says 172.21.15.27:60656) 2023-07-06T20:15:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6679c700 1 -- 172.21.15.27:0/2099104399 learned_addr learned my addr 172.21.15.27:0/2099104399 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6579a700 1 --2- 172.21.15.27:0/2099104399 >> v2:172.21.15.153:3300/0 conn(0x7f1e600b5c90 0x7f1e600d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e65f9b700 1 --2- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e64f99700 1 -- 172.21.15.27:0/2099104399 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167870409 0 0) 0x7f1e600cf780 con 0x7f1e600b6100 2023-07-06T20:15:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e64f99700 1 -- 172.21.15.27:0/2099104399 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c0035b0 con 0x7f1e600b6100 2023-07-06T20:15:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6579a700 1 -- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 msgr2=0x7f1e600d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6579a700 1 --2- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6579a700 1 -- 172.21.15.27:0/2099104399 >> v1:172.21.15.27:6789/0 conn(0x7f1e600b6100 legacy=0x7f1e600cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.996+0000 7f1e6579a700 1 -- 172.21.15.27:0/2099104399 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e600cda50 con 0x7f1e600b5c90 2023-07-06T20:15:33.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.997+0000 7f1e6579a700 1 --2- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e600d2d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009840 tx=0x7f1e50009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e849b0015a6123b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:33.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.997+0000 7f1e64f99700 1 -- 172.21.15.27:0/2099104399 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50011830 con 0x7f1e600b5c90 2023-07-06T20:15:33.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.997+0000 7f1e64f99700 1 -- 172.21.15.27:0/2099104399 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50011990 con 0x7f1e600b5c90 2023-07-06T20:15:33.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.997+0000 7f1e64f99700 1 -- 172.21.15.27:0/2099104399 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50011d00 con 0x7f1e600b5c90 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 -- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 msgr2=0x7f1e600d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 --2- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e600d2d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009840 tx=0x7f1e50009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 -- 172.21.15.27:0/2099104399 shutdown_connections 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 --2- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 --2- 172.21.15.27:0/2099104399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e600d2d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 -- 172.21.15.27:0/2099104399 >> 172.21.15.27:0/2099104399 conn(0x7f1e6006c150 msgr2=0x7f1e6006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:33.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 -- 172.21.15.27:0/2099104399 shutdown_connections 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.998+0000 7f1e681ff700 1 -- 172.21.15.27:0/2099104399 wait complete. 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:32.999+0000 7f1e681ff700 1 Processor -- start 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.000+0000 7f1e681ff700 1 -- start start 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e681ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e601502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e681ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 0x7f1e600c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e681ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e681ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601e6010 con 0x7f1e600b6100 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e681ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601507f0 con 0x7f1e600b5c90 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e681ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600ca960 con 0x7f1e6015fca0 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 0x7f1e600c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 0x7f1e600c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42228/0 (socket says 172.21.15.27:42228) 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 -- 172.21.15.27:0/1511744115 learned_addr learned my addr 172.21.15.27:0/1511744115 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e65f9b700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e601502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6679c700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 -- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 msgr2=0x7f1e600b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 -- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 msgr2=0x7f1e601502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e601502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.001+0000 7f1e6579a700 1 -- 172.21.15.27:0/1511744115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500094a0 con 0x7f1e600b6100 2023-07-06T20:15:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.002+0000 7f1e6579a700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 0x7f1e600c9f10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1e500090b0 tx=0x7f1e50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.002+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50004550 con 0x7f1e600b6100 2023-07-06T20:15:33.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.002+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5001f070 con 0x7f1e600b6100 2023-07-06T20:15:33.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.002+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500038c0 con 0x7f1e600b6100 2023-07-06T20:15:33.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.002+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e600cac40 con 0x7f1e600b6100 2023-07-06T20:15:33.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.002+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e600cb200 con 0x7f1e600b6100 2023-07-06T20:15:33.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.003+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e600cde30 con 0x7f1e600b6100 2023-07-06T20:15:33.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.004+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e50003a20 con 0x7f1e600b6100 2023-07-06T20:15:33.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.005+0000 7f1e567fc700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1e4406ad10 0x7f1e4406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.005+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e50021070 con 0x7f1e600b6100 2023-07-06T20:15:33.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.006+0000 7f1e65f9b700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1e4406ad10 0x7f1e4406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.007+0000 7f1e65f9b700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1e4406ad10 0x7f1e4406d1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e600d2250 tx=0x7f1e5c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:33.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.010+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e50055ca0 con 0x7f1e600b6100 2023-07-06T20:15:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.237+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e600cfb10 con 0x7f1e600b6100 2023-07-06T20:15:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.237+0000 7f1e567fc700 1 -- 172.21.15.27:0/1511744115 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1e5002e350 con 0x7f1e600b6100 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1e4406ad10 msgr2=0x7f1e4406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1e4406ad10 0x7f1e4406d1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e600d2250 tx=0x7f1e5c009210 comp rx=0 tx=0).stop 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 msgr2=0x7f1e600c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 0x7f1e600c9f10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1e500090b0 tx=0x7f1e50004750 comp rx=0 tx=0).stop 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 shutdown_connections 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1e4406ad10 0x7f1e4406d1c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e6015fca0 0x7f1e600b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e600b5c90 0x7f1e601502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 --2- 172.21.15.27:0/1511744115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e600b6100 0x7f1e600c9f10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 >> 172.21.15.27:0/1511744115 conn(0x7f1e6006c150 msgr2=0x7f1e6015a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 shutdown_connections 2023-07-06T20:15:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.240+0000 7f1e681ff700 1 -- 172.21.15.27:0/1511744115 wait complete. 2023-07-06T20:15:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-07-06T20:15:33.260 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-06T20:15:33.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:33.261 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-07-06T20:15:33.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:15:33.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:33.357 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T20:15:33.404 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:15:33.405 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:33.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345616k,nr_inodes=4086404,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T20:15:33.451 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:15:33.452 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:33.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T20:15:33.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:33.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.698+0000 7f7fa09ff700 1 Processor -- start 2023-07-06T20:15:33.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.699+0000 7f7fa09ff700 1 -- start start 2023-07-06T20:15:33.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7fa09ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f9c15fca0 0x7f7f9c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7fa09ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7f9c0b5c90 0x7f7f9c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7fa09ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0cf780 con 0x7f7f9c0b6110 2023-07-06T20:15:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7fa09ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cd490 con 0x7f7f9c15fca0 2023-07-06T20:15:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7fa09ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cd770 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7f9a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f9c15fca0 0x7f7f9c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7f9a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f9c15fca0 0x7f7f9c0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42244/0 (socket says 172.21.15.27:42244) 2023-07-06T20:15:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7f9a59c700 1 -- 172.21.15.27:0/2018009634 learned_addr learned my addr 172.21.15.27:0/2018009634 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.700+0000 7f7f93fff700 1 --2- 172.21.15.27:0/2018009634 >> v2:172.21.15.153:3300/0 conn(0x7f7f9c0b5c90 0x7f7f9c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f9a59c700 1 -- 172.21.15.27:0/2018009634 >> v2:172.21.15.153:3300/0 conn(0x7f7f9c0b5c90 msgr2=0x7f7f9c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f9a59c700 1 --2- 172.21.15.27:0/2018009634 >> v2:172.21.15.153:3300/0 conn(0x7f7f9c0b5c90 0x7f7f9c0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f9a59c700 1 -- 172.21.15.27:0/2018009634 >> v1:172.21.15.153:6789/0 conn(0x7f7f9c0b6110 legacy=0x7f7f9c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f9a59c700 1 -- 172.21.15.27:0/2018009634 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0cda50 con 0x7f7f9c15fca0 2023-07-06T20:15:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f9a59c700 1 --2- 172.21.15.27:0/2018009634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c15fca0 0x7f7f9c0d1420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c06b6f0 tx=0x7f7f8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bef62d952c9a6be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f99d9b700 1 -- 172.21.15.27:0/2018009634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00b870 con 0x7f7f9c15fca0 2023-07-06T20:15:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f99d9b700 1 -- 172.21.15.27:0/2018009634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00b9d0 con 0x7f7f9c15fca0 2023-07-06T20:15:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.701+0000 7f7f99d9b700 1 -- 172.21.15.27:0/2018009634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00bd60 con 0x7f7f9c15fca0 2023-07-06T20:15:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 -- 172.21.15.27:0/2018009634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c15fca0 msgr2=0x7f7f9c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/2018009634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c15fca0 0x7f7f9c0d1420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c06b6f0 tx=0x7f7f8c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 -- 172.21.15.27:0/2018009634 shutdown_connections 2023-07-06T20:15:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/2018009634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c15fca0 0x7f7f9c0d1420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/2018009634 >> v2:172.21.15.153:3300/0 conn(0x7f7f9c0b5c90 0x7f7f9c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 -- 172.21.15.27:0/2018009634 >> 172.21.15.27:0/2018009634 conn(0x7f7f9c06c150 msgr2=0x7f7f9c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 -- 172.21.15.27:0/2018009634 shutdown_connections 2023-07-06T20:15:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.702+0000 7f7fa09ff700 1 -- 172.21.15.27:0/2018009634 wait complete. 2023-07-06T20:15:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.704+0000 7f7fa09ff700 1 Processor -- start 2023-07-06T20:15:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.704+0000 7f7fa09ff700 1 -- start start 2023-07-06T20:15:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7fa09ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 0x7f7f9c0d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7fa09ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f9c0b6110 0x7f7f9c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7fa09ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f9c15fca0 0x7f7f9c152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7fa09ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c7000 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7fa09ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c7250 con 0x7f7f9c0b6110 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7fa09ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0c7530 con 0x7f7f9c15fca0 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 0x7f7f9c0d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 0x7f7f9c0d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42256/0 (socket says 172.21.15.27:42256) 2023-07-06T20:15:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 -- 172.21.15.27:0/965616762 learned_addr learned my addr 172.21.15.27:0/965616762 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f93fff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f9c0b6110 0x7f7f9c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 -- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f9c15fca0 msgr2=0x7f7f9c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9ad9d700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f9c15fca0 0x7f7f9c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f9c15fca0 0x7f7f9c152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 -- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f9c0b6110 msgr2=0x7f7f9c0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f9c0b6110 0x7f7f9c0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9a59c700 1 -- 172.21.15.27:0/965616762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009550 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f93fff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f9c0b6110 0x7f7f9c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.705+0000 7f7f9ad9d700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f9c15fca0 0x7f7f9c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.706+0000 7f7f9a59c700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 0x7f7f9c0d2dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c000c00 tx=0x7f7f8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.706+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c004520 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.706+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c01e070 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.706+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c0c7810 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.706+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0038a0 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.706+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c0c7dd0 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.707+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c0008d0 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.708+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f8c003a00 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.709+0000 7f7f92ffd700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7f7c06ace0 0x7f7f7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.709+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f8c020070 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.710+0000 7f7f93fff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7f7c06ace0 0x7f7f7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.710+0000 7f7f93fff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7f7c06ace0 0x7f7f7c06d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0c6b30 tx=0x7f7f84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.713+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f8c055b60 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.964+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f9c15a280 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.965+0000 7f7f92ffd700 1 -- 172.21.15.27:0/965616762 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f7f8c015090 con 0x7f7f9c0b5c90 2023-07-06T20:15:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:33.967 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:15:16.714161+0000","modified":"2023-07-06T20:15:20.237603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/827990525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":827990525},{"type":"v1","addr":"172.21.15.27:6835","nonce":827990525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T20:15:33.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7f7c06ace0 msgr2=0x7f7f7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7f7c06ace0 0x7f7f7c06d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0c6b30 tx=0x7f7f84008040 comp rx=0 tx=0).stop 2023-07-06T20:15:33.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 msgr2=0x7f7f9c0d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 0x7f7f9c0d2dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c000c00 tx=0x7f7f8c004750 comp rx=0 tx=0).stop 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 shutdown_connections 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7f7c06ace0 0x7f7f7c06d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7f9c15fca0 0x7f7f9c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f9c0b5c90 0x7f7f9c0d2dd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 --2- 172.21.15.27:0/965616762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f9c0b6110 0x7f7f9c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 >> 172.21.15.27:0/965616762 conn(0x7f7f9c06c150 msgr2=0x7f7f9c0b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 shutdown_connections 2023-07-06T20:15:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:33.967+0000 7f7fa09ff700 1 -- 172.21.15.27:0/965616762 wait complete. 2023-07-06T20:15:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2023-07-06T20:15:33.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:34.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.191+0000 7f51309ff700 1 Processor -- start 2023-07-06T20:15:34.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.191+0000 7f51309ff700 1 -- start start 2023-07-06T20:15:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.192+0000 7f51309ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f512c0b5c90 0x7f512c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.192+0000 7f51309ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f512c0b6110 0x7f512c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.192+0000 7f51309ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0cf780 con 0x7f512c15fca0 2023-07-06T20:15:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.192+0000 7f51309ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0cd490 con 0x7f512c0b5c90 2023-07-06T20:15:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.192+0000 7f51309ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0cd770 con 0x7f512c0b6110 2023-07-06T20:15:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f5129d9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f512c0b5c90 0x7f512c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f512a59c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f512c15fca0 0x7f512c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60666/0 (socket says 172.21.15.27:60666) 2023-07-06T20:15:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f512a59c700 1 -- 172.21.15.27:0/1339231890 learned_addr learned my addr 172.21.15.27:0/1339231890 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f512ad9d700 1 --2- 172.21.15.27:0/1339231890 >> v2:172.21.15.153:3300/0 conn(0x7f512c0b6110 0x7f512c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f512959a700 1 -- 172.21.15.27:0/1339231890 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922433962 0 0) 0x7f512c0cf780 con 0x7f512c15fca0 2023-07-06T20:15:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f5129d9b700 1 -- 172.21.15.27:0/1339231890 >> v2:172.21.15.153:3300/0 conn(0x7f512c0b6110 msgr2=0x7f512c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f5129d9b700 1 --2- 172.21.15.27:0/1339231890 >> v2:172.21.15.153:3300/0 conn(0x7f512c0b6110 0x7f512c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f5129d9b700 1 -- 172.21.15.27:0/1339231890 >> v1:172.21.15.27:6789/0 conn(0x7f512c15fca0 legacy=0x7f512c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f5129d9b700 1 -- 172.21.15.27:0/1339231890 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0cda50 con 0x7f512c0b5c90 2023-07-06T20:15:34.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.193+0000 7f5129d9b700 1 --2- 172.21.15.27:0/1339231890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0b5c90 0x7f512c0d2d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f512c0cd410 tx=0x7f5120009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdb6b8c5d7961d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f512959a700 1 -- 172.21.15.27:0/1339231890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120011830 con 0x7f512c0b5c90 2023-07-06T20:15:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f512959a700 1 -- 172.21.15.27:0/1339231890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120011990 con 0x7f512c0b5c90 2023-07-06T20:15:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f512959a700 1 -- 172.21.15.27:0/1339231890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120011d20 con 0x7f512c0b5c90 2023-07-06T20:15:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 -- 172.21.15.27:0/1339231890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0b5c90 msgr2=0x7f512c0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 --2- 172.21.15.27:0/1339231890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0b5c90 0x7f512c0d2d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f512c0cd410 tx=0x7f5120009b20 comp rx=0 tx=0).stop 2023-07-06T20:15:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 -- 172.21.15.27:0/1339231890 shutdown_connections 2023-07-06T20:15:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 --2- 172.21.15.27:0/1339231890 >> v2:172.21.15.153:3300/0 conn(0x7f512c0b6110 0x7f512c0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 --2- 172.21.15.27:0/1339231890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0b5c90 0x7f512c0d2d40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 -- 172.21.15.27:0/1339231890 >> 172.21.15.27:0/1339231890 conn(0x7f512c06c150 msgr2=0x7f512c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 -- 172.21.15.27:0/1339231890 shutdown_connections 2023-07-06T20:15:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.194+0000 7f51309ff700 1 -- 172.21.15.27:0/1339231890 wait complete. 2023-07-06T20:15:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.196+0000 7f51309ff700 1 Processor -- start 2023-07-06T20:15:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.196+0000 7f51309ff700 1 -- start start 2023-07-06T20:15:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f51309ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f512c0b5c90 0x7f512c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f51309ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c0b6110 0x7f512c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f51309ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 0x7f512c0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f51309ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0c6170 con 0x7f512c15fca0 2023-07-06T20:15:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f51309ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0cb830 con 0x7f512c0b6110 2023-07-06T20:15:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f51309ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0cbb10 con 0x7f512c0b5c90 2023-07-06T20:15:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f512ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 0x7f512c0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f512ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 0x7f512c0ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42268/0 (socket says 172.21.15.27:42268) 2023-07-06T20:15:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.197+0000 7f512ad9d700 1 -- 172.21.15.27:0/2488587355 learned_addr learned my addr 172.21.15.27:0/2488587355 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f5129d9b700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c0b6110 0x7f512c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512a59c700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f512c0b5c90 0x7f512c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512ad9d700 1 -- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f512c0b5c90 msgr2=0x7f512c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512ad9d700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f512c0b5c90 0x7f512c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512ad9d700 1 -- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c0b6110 msgr2=0x7f512c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512ad9d700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c0b6110 0x7f512c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512ad9d700 1 -- 172.21.15.27:0/2488587355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51200094a0 con 0x7f512c15fca0 2023-07-06T20:15:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512a59c700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f512c0b5c90 0x7f512c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f5129d9b700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c0b6110 0x7f512c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f512ad9d700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 0x7f512c0ca8e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f512400efc0 tx=0x7f512400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124009ea0 con 0x7f512c15fca0 2023-07-06T20:15:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400f040 con 0x7f512c15fca0 2023-07-06T20:15:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0cbdf0 con 0x7f512c15fca0 2023-07-06T20:15:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.198+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124014480 con 0x7f512c15fca0 2023-07-06T20:15:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.199+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0cc3b0 con 0x7f512c15fca0 2023-07-06T20:15:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.200+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c0008d0 con 0x7f512c15fca0 2023-07-06T20:15:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.201+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51240145e0 con 0x7f512c15fca0 2023-07-06T20:15:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.201+0000 7f511affd700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f510c06ace0 0x7f510c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.202+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5124089e50 con 0x7f512c15fca0 2023-07-06T20:15:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.202+0000 7f512a59c700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f510c06ace0 0x7f510c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.202+0000 7f512a59c700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f510c06ace0 0x7f510c06d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f511c000f60 tx=0x7f511c007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.206+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5124054e60 con 0x7f512c15fca0 2023-07-06T20:15:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.416+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f512c0d06b0 con 0x7f512c15fca0 2023-07-06T20:15:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.417+0000 7f511affd700 1 -- 172.21.15.27:0/2488587355 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f5124058480 con 0x7f512c15fca0 2023-07-06T20:15:34.419 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:34.420 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:18.212485+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f510c06ace0 msgr2=0x7f510c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f510c06ace0 0x7f510c06d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f511c000f60 tx=0x7f511c007d30 comp rx=0 tx=0).stop 2023-07-06T20:15:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 msgr2=0x7f512c0ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 0x7f512c0ca8e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f512400efc0 tx=0x7f512400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 shutdown_connections 2023-07-06T20:15:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f510c06ace0 0x7f510c06d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c15fca0 0x7f512c0ca8e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f512c0b5c90 0x7f512c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 --2- 172.21.15.27:0/2488587355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c0b6110 0x7f512c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 >> 172.21.15.27:0/2488587355 conn(0x7f512c06c150 msgr2=0x7f512c0cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 shutdown_connections 2023-07-06T20:15:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.420+0000 7f51309ff700 1 -- 172.21.15.27:0/2488587355 wait complete. 2023-07-06T20:15:34.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:44.411793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T20:13:51.018054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:15:15.163204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:15:15.707733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:15:34.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:15:34.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.646+0000 7fee999ff700 1 Processor -- start 2023-07-06T20:15:34.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.646+0000 7fee999ff700 1 -- start start 2023-07-06T20:15:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.647+0000 7fee999ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee9415fca0 0x7fee940d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.647+0000 7fee999ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee940cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.647+0000 7fee999ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee940cf780 con 0x7fee940b5c90 2023-07-06T20:15:34.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.647+0000 7fee999ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee940cd490 con 0x7fee9415fca0 2023-07-06T20:15:34.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee999ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee940cd770 con 0x7fee940b6110 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee937fe700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee940cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee92ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee9415fca0 0x7fee940d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee927fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fee940b5c90 0x7fee940d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48438/0 (socket says 172.21.15.27:48438) 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee927fc700 1 -- 172.21.15.27:0/1337075397 learned_addr learned my addr 172.21.15.27:0/1337075397 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee937fe700 1 -- 172.21.15.27:0/1337075397 >> v2:172.21.15.153:3300/0 conn(0x7fee9415fca0 msgr2=0x7fee940d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee937fe700 1 --2- 172.21.15.27:0/1337075397 >> v2:172.21.15.153:3300/0 conn(0x7fee9415fca0 0x7fee940d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee937fe700 1 -- 172.21.15.27:0/1337075397 >> v1:172.21.15.153:6789/0 conn(0x7fee940b5c90 legacy=0x7fee940d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee91ffb700 1 -- 172.21.15.27:0/1337075397 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130860213 0 0) 0x7fee940cf780 con 0x7fee940b5c90 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee937fe700 1 -- 172.21.15.27:0/1337075397 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940cda50 con 0x7fee940b6110 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.648+0000 7fee92ffd700 1 --2- 172.21.15.27:0/1337075397 >> v2:172.21.15.153:3300/0 conn(0x7fee9415fca0 0x7fee940d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.649+0000 7fee937fe700 1 --2- 172.21.15.27:0/1337075397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee940cfc10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009d90 tx=0x7fee8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6434e19a0ac76f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.649+0000 7fee91ffb700 1 -- 172.21.15.27:0/1337075397 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00e070 con 0x7fee940b6110 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.649+0000 7fee91ffb700 1 -- 172.21.15.27:0/1337075397 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c00bb20 con 0x7fee940b6110 2023-07-06T20:15:34.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.649+0000 7fee91ffb700 1 -- 172.21.15.27:0/1337075397 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c012430 con 0x7fee940b6110 2023-07-06T20:15:34.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.649+0000 7fee999ff700 1 -- 172.21.15.27:0/1337075397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 msgr2=0x7fee940cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.649+0000 7fee999ff700 1 --2- 172.21.15.27:0/1337075397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee940cfc10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009d90 tx=0x7fee8c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.650+0000 7fee999ff700 1 -- 172.21.15.27:0/1337075397 shutdown_connections 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.650+0000 7fee999ff700 1 --2- 172.21.15.27:0/1337075397 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee940cfc10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.650+0000 7fee999ff700 1 --2- 172.21.15.27:0/1337075397 >> v2:172.21.15.153:3300/0 conn(0x7fee9415fca0 0x7fee940d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.650+0000 7fee999ff700 1 -- 172.21.15.27:0/1337075397 >> 172.21.15.27:0/1337075397 conn(0x7fee9406c150 msgr2=0x7fee9406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.650+0000 7fee999ff700 1 -- 172.21.15.27:0/1337075397 shutdown_connections 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.650+0000 7fee999ff700 1 -- 172.21.15.27:0/1337075397 wait complete. 2023-07-06T20:15:34.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.651+0000 7fee999ff700 1 Processor -- start 2023-07-06T20:15:34.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.652+0000 7fee999ff700 1 -- start start 2023-07-06T20:15:34.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee999ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 0x7fee941e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee999ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee941f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee999ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9415fca0 0x7fee9414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee999ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee941f6860 con 0x7fee9415fca0 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee999ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940b5120 con 0x7fee940b5c90 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee999ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee940b2ab0 con 0x7fee940b6110 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 0x7fee941e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 0x7fee941e5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.27:59938/0 (socket says 172.21.15.27:59938) 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9415fca0 0x7fee9414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 -- 172.21.15.27:0/4042468323 learned_addr learned my addr 172.21.15.27:0/4042468323 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee927fc700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee941f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 -- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 msgr2=0x7fee941f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee941f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 -- 172.21.15.27:0/4042468323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9415fca0 msgr2=0x7fee9414b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9415fca0 0x7fee9414b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee92ffd700 1 -- 172.21.15.27:0/4042468323 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840094a0 con 0x7fee940b5c90 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.653+0000 7fee937fe700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9415fca0 0x7fee9414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.654+0000 7fee92ffd700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 0x7fee941e5ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fee8400ec10 tx=0x7fee8400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.654+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84009760 con 0x7fee940b5c90 2023-07-06T20:15:34.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.654+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8400c850 con 0x7fee940b5c90 2023-07-06T20:15:34.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.654+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400cbe0 con 0x7fee940b5c90 2023-07-06T20:15:34.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.654+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c009a70 con 0x7fee940b5c90 2023-07-06T20:15:34.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.654+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee940b5650 con 0x7fee940b5c90 2023-07-06T20:15:34.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.655+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee940b3070 con 0x7fee940b5c90 2023-07-06T20:15:34.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.656+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee8400cd40 con 0x7fee940b5c90 2023-07-06T20:15:34.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.657+0000 7fee7bfff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fee8006ace0 0x7fee8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.657+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee840895a0 con 0x7fee940b5c90 2023-07-06T20:15:34.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.658+0000 7fee927fc700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fee8006ace0 0x7fee8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.658+0000 7fee927fc700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fee8006ace0 0x7fee8006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee940b4d10 tx=0x7fee88007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.661+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee840580c0 con 0x7fee940b5c90 2023-07-06T20:15:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.866+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee940008d0 con 0x7fee940b5c90 2023-07-06T20:15:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.867+0000 7fee7bfff700 1 -- 172.21.15.27:0/4042468323 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee840210a0 con 0x7fee940b5c90 2023-07-06T20:15:34.869 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fee8006ace0 msgr2=0x7fee8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fee8006ace0 0x7fee8006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee940b4d10 tx=0x7fee88007de0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 msgr2=0x7fee941e5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 0x7fee941e5ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fee8400ec10 tx=0x7fee8400ef20 comp rx=0 tx=0).stop 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 shutdown_connections 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fee8006ace0 0x7fee8006d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9415fca0 0x7fee9414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee940b5c90 0x7fee941e5ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 --2- 172.21.15.27:0/4042468323 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fee940b6110 0x7fee941f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 >> 172.21.15.27:0/4042468323 conn(0x7fee9406c150 msgr2=0x7fee940d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 shutdown_connections 2023-07-06T20:15:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:34.870+0000 7fee999ff700 1 -- 172.21.15.27:0/4042468323 wait complete. 2023-07-06T20:15:34.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T20:15:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.095+0000 7f89d37ff700 1 Processor -- start 2023-07-06T20:15:35.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.095+0000 7f89d37ff700 1 -- start start 2023-07-06T20:15:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.096+0000 7f89d37ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89cc15fca0 0x7f89cc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.096+0000 7f89d37ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.096+0000 7f89d37ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f89cc0b60e0 0x7f89cc0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.096+0000 7f89d37ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0cf780 con 0x7f89cc0b60e0 2023-07-06T20:15:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.096+0000 7f89d37ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0cd490 con 0x7f89cc15fca0 2023-07-06T20:15:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.096+0000 7f89d37ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0cd770 con 0x7f89cc0b5c90 2023-07-06T20:15:35.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f89cc0b60e0 0x7f89cc0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d159b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89cc15fca0 0x7f89cc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f89cc0b60e0 0x7f89cc0d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42284/0 (socket says 172.21.15.27:42284) 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 -- 172.21.15.27:0/1540418086 learned_addr learned my addr 172.21.15.27:0/1540418086 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d0d9a700 1 --2- 172.21.15.27:0/1540418086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 -- 172.21.15.27:0/1540418086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 msgr2=0x7f89cc0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 --2- 172.21.15.27:0/1540418086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 -- 172.21.15.27:0/1540418086 >> v2:172.21.15.153:3300/0 conn(0x7f89cc15fca0 msgr2=0x7f89cc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:35.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 --2- 172.21.15.27:0/1540418086 >> v2:172.21.15.153:3300/0 conn(0x7f89cc15fca0 0x7f89cc0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.097+0000 7f89d1d9c700 1 -- 172.21.15.27:0/1540418086 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0cda50 con 0x7f89cc0b60e0 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.098+0000 7f89d1d9c700 1 --2- 172.21.15.27:0/1540418086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0d3280 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0d0930 tx=0x7f89c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab08e580fe297a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.098+0000 7f89c3fff700 1 -- 172.21.15.27:0/1540418086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400e070 con 0x7f89cc0b60e0 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.098+0000 7f89c3fff700 1 -- 172.21.15.27:0/1540418086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400b910 con 0x7f89cc0b60e0 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.098+0000 7f89c3fff700 1 -- 172.21.15.27:0/1540418086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400bcc0 con 0x7f89cc0b60e0 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.098+0000 7f89d37ff700 1 -- 172.21.15.27:0/1540418086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc0b60e0 msgr2=0x7f89cc0d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.098+0000 7f89d37ff700 1 --2- 172.21.15.27:0/1540418086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0d3280 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0d0930 tx=0x7f89c400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 -- 172.21.15.27:0/1540418086 shutdown_connections 2023-07-06T20:15:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 --2- 172.21.15.27:0/1540418086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0d3280 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 --2- 172.21.15.27:0/1540418086 >> v2:172.21.15.153:3300/0 conn(0x7f89cc15fca0 0x7f89cc0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 --2- 172.21.15.27:0/1540418086 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 -- 172.21.15.27:0/1540418086 >> 172.21.15.27:0/1540418086 conn(0x7f89cc06c150 msgr2=0x7f89cc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 -- 172.21.15.27:0/1540418086 shutdown_connections 2023-07-06T20:15:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.099+0000 7f89d37ff700 1 -- 172.21.15.27:0/1540418086 wait complete. 2023-07-06T20:15:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.100+0000 7f89d37ff700 1 Processor -- start 2023-07-06T20:15:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.101+0000 7f89d37ff700 1 -- start start 2023-07-06T20:15:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d37ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d37ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d37ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 0x7f89cc0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d37ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0b49e0 con 0x7f89cc15fca0 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d37ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0b3400 con 0x7f89cc0b60e0 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d37ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0b36e0 con 0x7f89cc0b5c90 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 0x7f89cc0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 0x7f89cc0cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42290/0 (socket says 172.21.15.27:42290) 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 -- 172.21.15.27:0/3598915791 learned_addr learned my addr 172.21.15.27:0/3598915791 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d159b700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d0d9a700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 -- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 msgr2=0x7f89cc156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 -- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc0b60e0 msgr2=0x7f89cc0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.102+0000 7f89d1d9c700 1 -- 172.21.15.27:0/3598915791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009b30 con 0x7f89cc15fca0 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.103+0000 7f89d1d9c700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 0x7f89cc0cb2c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f89c4012000 tx=0x7f89c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.103+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40099b0 con 0x7f89cc15fca0 2023-07-06T20:15:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.103+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c400ce80 con 0x7f89cc15fca0 2023-07-06T20:15:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.103+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4004490 con 0x7f89cc15fca0 2023-07-06T20:15:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.103+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc0b39c0 con 0x7f89cc15fca0 2023-07-06T20:15:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.103+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc0b3f80 con 0x7f89cc15fca0 2023-07-06T20:15:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.104+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc0b4170 con 0x7f89cc15fca0 2023-07-06T20:15:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.105+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f89c40045f0 con 0x7f89cc15fca0 2023-07-06T20:15:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.106+0000 7f89c1ffb700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f89b006ace0 0x7f89b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.106+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89c408a770 con 0x7f89cc15fca0 2023-07-06T20:15:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.106+0000 7f89d159b700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f89b006ace0 0x7f89b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.107+0000 7f89d159b700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f89b006ace0 0x7f89b006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009b40 tx=0x7f89c8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.110+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f89c4055810 con 0x7f89cc15fca0 2023-07-06T20:15:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.319+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89cc0008d0 con 0x7f89cc15fca0 2023-07-06T20:15:35.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.319+0000 7f89c1ffb700 1 -- 172.21.15.27:0/3598915791 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89c4058e30 con 0x7f89cc15fca0 2023-07-06T20:15:35.321 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 8 2023-07-06T20:15:35.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f89b006ace0 msgr2=0x7f89b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f89b006ace0 0x7f89b006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009b40 tx=0x7f89c8009210 comp rx=0 tx=0).stop 2023-07-06T20:15:35.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 msgr2=0x7f89cc0cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 0x7f89cc0cb2c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f89c4012000 tx=0x7f89c4004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 shutdown_connections 2023-07-06T20:15:35.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f89b006ace0 0x7f89b006d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89cc15fca0 0x7f89cc0cb2c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89cc0b5c90 0x7f89cc156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 --2- 172.21.15.27:0/3598915791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89cc0b60e0 0x7f89cc0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 >> 172.21.15.27:0/3598915791 conn(0x7f89cc06c150 msgr2=0x7f89cc0b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 shutdown_connections 2023-07-06T20:15:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.322+0000 7f89d37ff700 1 -- 172.21.15.27:0/3598915791 wait complete. 2023-07-06T20:15:35.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:15:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.547+0000 7ff04c9ff700 1 Processor -- start 2023-07-06T20:15:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.547+0000 7ff04c9ff700 1 -- start start 2023-07-06T20:15:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.548+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff0480d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.548+0000 7ff04c9ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0480b60e0 0x7ff0480d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.548+0000 7ff04c9ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480cf780 con 0x7ff04815fca0 2023-07-06T20:15:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.548+0000 7ff04c9ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0480cd490 con 0x7ff0480b60e0 2023-07-06T20:15:35.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.548+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480cd770 con 0x7ff0480b5c90 2023-07-06T20:15:35.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff04659c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff04815fca0 0x7ff0480d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60670/0 (socket says 172.21.15.27:60670) 2023-07-06T20:15:35.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff04659c700 1 -- 172.21.15.27:0/2123030135 learned_addr learned my addr 172.21.15.27:0/2123030135 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff046d9d700 1 --2- 172.21.15.27:0/2123030135 >> v2:172.21.15.153:3300/0 conn(0x7ff0480b60e0 0x7ff0480d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff045d9b700 1 --2- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff0480d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff04559a700 1 -- 172.21.15.27:0/2123030135 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756126713 0 0) 0x7ff0480cf780 con 0x7ff04815fca0 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff04559a700 1 -- 172.21.15.27:0/2123030135 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0300035b0 con 0x7ff04815fca0 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff046d9d700 1 -- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 msgr2=0x7ff0480d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff046d9d700 1 --2- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff0480d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff046d9d700 1 -- 172.21.15.27:0/2123030135 >> v1:172.21.15.27:6789/0 conn(0x7ff04815fca0 legacy=0x7ff0480d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.549+0000 7ff046d9d700 1 -- 172.21.15.27:0/2123030135 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480cda50 con 0x7ff0480b60e0 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff046d9d700 1 --2- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff0480d3280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff040009d90 tx=0x7ff04000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df5279b6c2234976 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04559a700 1 -- 172.21.15.27:0/2123030135 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000e070 con 0x7ff0480b60e0 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04559a700 1 -- 172.21.15.27:0/2123030135 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000bb20 con 0x7ff0480b60e0 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04559a700 1 -- 172.21.15.27:0/2123030135 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040012430 con 0x7ff0480b60e0 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04c9ff700 1 -- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 msgr2=0x7ff0480d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff0480d3280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff040009d90 tx=0x7ff04000b330 comp rx=0 tx=0).stop 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04c9ff700 1 -- 172.21.15.27:0/2123030135 shutdown_connections 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff0480d3280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/2123030135 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff0480d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.550+0000 7ff04c9ff700 1 -- 172.21.15.27:0/2123030135 >> 172.21.15.27:0/2123030135 conn(0x7ff04806c150 msgr2=0x7ff04806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:35.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.551+0000 7ff04c9ff700 1 -- 172.21.15.27:0/2123030135 shutdown_connections 2023-07-06T20:15:35.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.551+0000 7ff04c9ff700 1 -- 172.21.15.27:0/2123030135 wait complete. 2023-07-06T20:15:35.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.552+0000 7ff04c9ff700 1 Processor -- start 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.552+0000 7ff04c9ff700 1 -- start start 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.553+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff04815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.553+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff048157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.553+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 0x7ff0480b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.553+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04815bbd0 con 0x7ff04815fca0 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.553+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480d2c00 con 0x7ff0480b60e0 2023-07-06T20:15:35.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.553+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480d2ee0 con 0x7ff0480b5c90 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 0x7ff0480b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff04659c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff04815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 0x7ff0480b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42298/0 (socket says 172.21.15.27:42298) 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 -- 172.21.15.27:0/4026062290 learned_addr learned my addr 172.21.15.27:0/4026062290 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff045d9b700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff048157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 -- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 msgr2=0x7ff04815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff04815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 -- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 msgr2=0x7ff048157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff048157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.554+0000 7ff046d9d700 1 -- 172.21.15.27:0/4026062290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009a70 con 0x7ff04815fca0 2023-07-06T20:15:35.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.555+0000 7ff046d9d700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 0x7ff0480b3340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff040014040 tx=0x7ff040004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:35.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.555+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400129f0 con 0x7ff04815fca0 2023-07-06T20:15:35.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.555+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff040020070 con 0x7ff04815fca0 2023-07-06T20:15:35.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.555+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0480d31c0 con 0x7ff04815fca0 2023-07-06T20:15:35.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.555+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040012db0 con 0x7ff04815fca0 2023-07-06T20:15:35.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.555+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04815aab0 con 0x7ff04815fca0 2023-07-06T20:15:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.556+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0480008d0 con 0x7ff04815fca0 2023-07-06T20:15:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.557+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff040004040 con 0x7ff04815fca0 2023-07-06T20:15:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.558+0000 7ff036ffd700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff02806ace0 0x7ff02806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.558+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff040022070 con 0x7ff04815fca0 2023-07-06T20:15:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.558+0000 7ff04659c700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff02806ace0 0x7ff02806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:35.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.559+0000 7ff04659c700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff02806ace0 0x7ff02806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff038007fd0 tx=0x7ff038007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:35.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.562+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff040056130 con 0x7ff04815fca0 2023-07-06T20:15:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.768+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0480d3b80 con 0x7ff04815fca0 2023-07-06T20:15:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.769+0000 7ff036ffd700 1 -- 172.21.15.27:0/4026062290 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff04002c0a0 con 0x7ff04815fca0 2023-07-06T20:15:35.771 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-07-06T20:15:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff02806ace0 msgr2=0x7ff02806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff02806ace0 0x7ff02806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff038007fd0 tx=0x7ff038007dc0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 msgr2=0x7ff0480b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 0x7ff0480b3340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff040014040 tx=0x7ff040004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 shutdown_connections 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff02806ace0 0x7ff02806d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff04815fca0 0x7ff0480b3340 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff0480b5c90 0x7ff04815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 --2- 172.21.15.27:0/4026062290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0480b60e0 0x7ff048157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 >> 172.21.15.27:0/4026062290 conn(0x7ff04806c150 msgr2=0x7ff0480b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 shutdown_connections 2023-07-06T20:15:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.771+0000 7ff04c9ff700 1 -- 172.21.15.27:0/4026062290 wait complete. 2023-07-06T20:15:35.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:15:35.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.997+0000 7f64cdfff700 1 Processor -- start 2023-07-06T20:15:35.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.997+0000 7f64cdfff700 1 -- start start 2023-07-06T20:15:36.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64cdfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c80b5c90 0x7f64c80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64cdfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64c80b60e0 0x7f64c80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64cdfff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c80cf780 con 0x7f64c815fca0 2023-07-06T20:15:36.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64cdfff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c80cd490 con 0x7f64c80b60e0 2023-07-06T20:15:36.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64cdfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80cd770 con 0x7f64c80b5c90 2023-07-06T20:15:36.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64c7fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64c80b60e0 0x7f64c80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64c7fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64c80b60e0 0x7f64c80d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42312/0 (socket says 172.21.15.27:42312) 2023-07-06T20:15:36.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.998+0000 7f64c7fff700 1 -- 172.21.15.27:0/2179162631 learned_addr learned my addr 172.21.15.27:0/2179162631 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:36.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c6ffd700 1 --2- 172.21.15.27:0/2179162631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c80b5c90 0x7f64c80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c7fff700 1 -- 172.21.15.27:0/2179162631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c80b5c90 msgr2=0x7f64c80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:36.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c7fff700 1 --2- 172.21.15.27:0/2179162631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c80b5c90 0x7f64c80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c7fff700 1 -- 172.21.15.27:0/2179162631 >> v1:172.21.15.153:6789/0 conn(0x7f64c815fca0 legacy=0x7f64c80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c7fff700 1 -- 172.21.15.27:0/2179162631 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80cda50 con 0x7f64c80b60e0 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c7fff700 1 --2- 172.21.15.27:0/2179162631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b60e0 0x7f64c80d3280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009e50 tx=0x7f64c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d849b539bd0e7f1d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c67fc700 1 -- 172.21.15.27:0/2179162631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000e070 con 0x7f64c80b60e0 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c67fc700 1 -- 172.21.15.27:0/2179162631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c000b9c0 con 0x7f64c80b60e0 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:35.999+0000 7f64c67fc700 1 -- 172.21.15.27:0/2179162631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000bd70 con 0x7f64c80b60e0 2023-07-06T20:15:36.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 -- 172.21.15.27:0/2179162631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b60e0 msgr2=0x7f64c80d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 --2- 172.21.15.27:0/2179162631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b60e0 0x7f64c80d3280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009e50 tx=0x7f64c000b330 comp rx=0 tx=0).stop 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 -- 172.21.15.27:0/2179162631 shutdown_connections 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 --2- 172.21.15.27:0/2179162631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b60e0 0x7f64c80d3280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 --2- 172.21.15.27:0/2179162631 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c80b5c90 0x7f64c80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 -- 172.21.15.27:0/2179162631 >> 172.21.15.27:0/2179162631 conn(0x7f64c806c150 msgr2=0x7f64c806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 -- 172.21.15.27:0/2179162631 shutdown_connections 2023-07-06T20:15:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.000+0000 7f64cdfff700 1 -- 172.21.15.27:0/2179162631 wait complete. 2023-07-06T20:15:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.002+0000 7f64cdfff700 1 Processor -- start 2023-07-06T20:15:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.002+0000 7f64cdfff700 1 -- start start 2023-07-06T20:15:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64cdfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 0x7f64c8153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64cdfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c80b60e0 0x7f64c815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64cdfff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c815fca0 0x7f64c81f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64cdfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8157cd0 con 0x7f64c80b5c90 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64cdfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8153f40 con 0x7f64c80b60e0 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64cdfff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c815c650 con 0x7f64c815fca0 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 0x7f64c8153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 0x7f64c8153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42318/0 (socket says 172.21.15.27:42318) 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64c77fe700 1 -- 172.21.15.27:0/3533287790 learned_addr learned my addr 172.21.15.27:0/3533287790 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64c7fff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c815fca0 0x7f64c81f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.003+0000 7f64c6ffd700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c80b60e0 0x7f64c815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64c77fe700 1 -- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c815fca0 msgr2=0x7f64c81f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64c77fe700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c815fca0 0x7f64c81f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64c77fe700 1 -- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c80b60e0 msgr2=0x7f64c815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64c77fe700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c80b60e0 0x7f64c815a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64c77fe700 1 -- 172.21.15.27:0/3533287790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c0009b30 con 0x7f64c80b5c90 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64c77fe700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 0x7f64c8153a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c450 tx=0x7f64b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80052d0 con 0x7f64c80b5c90 2023-07-06T20:15:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b8005430 con 0x7f64c80b5c90 2023-07-06T20:15:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c815c930 con 0x7f64c80b5c90 2023-07-06T20:15:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80038c0 con 0x7f64c80b5c90 2023-07-06T20:15:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.004+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c81581c0 con 0x7f64c80b5c90 2023-07-06T20:15:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.005+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c80008d0 con 0x7f64c80b5c90 2023-07-06T20:15:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.006+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64b8003a20 con 0x7f64c80b5c90 2023-07-06T20:15:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.007+0000 7f64affff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f64b406ace0 0x7f64b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.007+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64b80888b0 con 0x7f64c80b5c90 2023-07-06T20:15:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.008+0000 7f64c6ffd700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f64b406ace0 0x7f64b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.009+0000 7f64c6ffd700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f64b406ace0 0x7f64b406d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64c815a210 tx=0x7f64bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.012+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f64b80538d0 con 0x7f64c80b5c90 2023-07-06T20:15:36.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.218+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64c80b2a60 con 0x7f64c80b5c90 2023-07-06T20:15:36.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.218+0000 7f64affff700 1 -- 172.21.15.27:0/3533287790 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64b8056ef0 con 0x7f64c80b5c90 2023-07-06T20:15:36.220 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-07-06T20:15:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f64b406ace0 msgr2=0x7f64b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f64b406ace0 0x7f64b406d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64c815a210 tx=0x7f64bc008040 comp rx=0 tx=0).stop 2023-07-06T20:15:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 msgr2=0x7f64c8153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 0x7f64c8153a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c450 tx=0x7f64b800c810 comp rx=0 tx=0).stop 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 shutdown_connections 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f64b406ace0 0x7f64b406d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f64c815fca0 0x7f64c81f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c80b5c90 0x7f64c8153a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 --2- 172.21.15.27:0/3533287790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c80b60e0 0x7f64c815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.220+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 >> 172.21.15.27:0/3533287790 conn(0x7f64c806c150 msgr2=0x7f64c80b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:36.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.221+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 shutdown_connections 2023-07-06T20:15:36.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.221+0000 7f64cdfff700 1 -- 172.21.15.27:0/3533287790 wait complete. 2023-07-06T20:15:36.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:36.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.443+0000 7fa7dc7ff700 1 Processor -- start 2023-07-06T20:15:36.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.444+0000 7fa7dc7ff700 1 -- start start 2023-07-06T20:15:36.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7dc7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa7d415fca0 0x7fa7d40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7dc7ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d40cf780 con 0x7fa7d40b5c90 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7dc7ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d40cd490 con 0x7fa7d40b6110 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7dc7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40cd770 con 0x7fa7d415fca0 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7da59b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa7d415fca0 0x7fa7d40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7da59b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa7d415fca0 0x7fa7d40d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42322/0 (socket says 172.21.15.27:42322) 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.445+0000 7fa7da59b700 1 -- 172.21.15.27:0/1363028030 learned_addr learned my addr 172.21.15.27:0/1363028030 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7da59b700 1 -- 172.21.15.27:0/1363028030 >> v1:172.21.15.153:6789/0 conn(0x7fa7d40b6110 legacy=0x7fa7d40cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:36.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7da59b700 1 -- 172.21.15.27:0/1363028030 >> v1:172.21.15.27:6789/0 conn(0x7fa7d40b5c90 legacy=0x7fa7d40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7da59b700 1 -- 172.21.15.27:0/1363028030 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40cda50 con 0x7fa7d415fca0 2023-07-06T20:15:36.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7d9599700 1 -- 172.21.15.27:0/1363028030 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271776948 0 0) 0x7fa7d40cf780 con 0x7fa7d40b5c90 2023-07-06T20:15:36.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7da59b700 1 --2- 172.21.15.27:0/1363028030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d415fca0 0x7fa7d40d1420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00098f0 tx=0x7fa7d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed0f6565675d9c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7d9599700 1 -- 172.21.15.27:0/1363028030 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004230 con 0x7fa7d415fca0 2023-07-06T20:15:36.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7d9599700 1 -- 172.21.15.27:0/1363028030 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d0004390 con 0x7fa7d415fca0 2023-07-06T20:15:36.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.446+0000 7fa7d9599700 1 -- 172.21.15.27:0/1363028030 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00036a0 con 0x7fa7d415fca0 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/1363028030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d415fca0 msgr2=0x7fa7d40d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/1363028030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d415fca0 0x7fa7d40d1420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00098f0 tx=0x7fa7d0009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/1363028030 shutdown_connections 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/1363028030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d415fca0 0x7fa7d40d1420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/1363028030 >> 172.21.15.27:0/1363028030 conn(0x7fa7d406c150 msgr2=0x7fa7d406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/1363028030 shutdown_connections 2023-07-06T20:15:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.447+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/1363028030 wait complete. 2023-07-06T20:15:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.449+0000 7fa7dc7ff700 1 Processor -- start 2023-07-06T20:15:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.449+0000 7fa7dc7ff700 1 -- start start 2023-07-06T20:15:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7dc7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7d40b5c90 0x7fa7d41f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7dc7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 0x7fa7d41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7dc7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7d415fca0 0x7fa7d4145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7dc7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41e6de0 con 0x7fa7d40b6110 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7dc7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41f2f80 con 0x7fa7d415fca0 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7dc7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4147e00 con 0x7fa7d40b5c90 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7d9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 0x7fa7d41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7d9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 0x7fa7d41596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42324/0 (socket says 172.21.15.27:42324) 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.450+0000 7fa7d9d9a700 1 -- 172.21.15.27:0/457324239 learned_addr learned my addr 172.21.15.27:0/457324239 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7da59b700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7d40b5c90 0x7fa7d41f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7dad9c700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7d415fca0 0x7fa7d4145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7d9d9a700 1 -- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7d40b5c90 msgr2=0x7fa7d41f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7d9d9a700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7d40b5c90 0x7fa7d41f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7d9d9a700 1 -- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7d415fca0 msgr2=0x7fa7d4145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7d9d9a700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7d415fca0 0x7fa7d4145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7d9d9a700 1 -- 172.21.15.27:0/457324239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d0009550 con 0x7fa7d40b6110 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7d9d9a700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 0x7fa7d41596c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c550 tx=0x7fa7c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40054e0 con 0x7fa7d40b6110 2023-07-06T20:15:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c4003680 con 0x7fa7d40b6110 2023-07-06T20:15:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d41e61e0 con 0x7fa7d40b6110 2023-07-06T20:15:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4003a30 con 0x7fa7d40b6110 2023-07-06T20:15:36.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.451+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d41f5b40 con 0x7fa7d40b6110 2023-07-06T20:15:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.453+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d40008d0 con 0x7fa7d40b6110 2023-07-06T20:15:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.454+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7c4003b90 con 0x7fa7d40b6110 2023-07-06T20:15:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.455+0000 7fa7caffd700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa7b806ace0 0x7fa7b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.455+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7c40889e0 con 0x7fa7d40b6110 2023-07-06T20:15:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.455+0000 7fa7da59b700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa7b806ace0 0x7fa7b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.456+0000 7fa7da59b700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa7b806ace0 0x7fa7b806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009130 tx=0x7fa7d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.459+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7c40539f0 con 0x7fa7d40b6110 2023-07-06T20:15:36.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.709+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7d4147f40 con 0x7fa7d40b6110 2023-07-06T20:15:36.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.710+0000 7fa7caffd700 1 -- 172.21.15.27:0/457324239 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fa7c4017020 con 0x7fa7d40b6110 2023-07-06T20:15:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:36.712 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:15:16.714161+0000","modified":"2023-07-06T20:15:20.237603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/827990525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":827990525},{"type":"v1","addr":"172.21.15.27:6835","nonce":827990525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa7b806ace0 msgr2=0x7fa7b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa7b806ace0 0x7fa7b806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009130 tx=0x7fa7d0008030 comp rx=0 tx=0).stop 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 msgr2=0x7fa7d41596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 0x7fa7d41596c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c550 tx=0x7fa7c400c910 comp rx=0 tx=0).stop 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 shutdown_connections 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fa7b806ace0 0x7fa7b806d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7d415fca0 0x7fa7d4145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7d40b5c90 0x7fa7d41f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 --2- 172.21.15.27:0/457324239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7d40b6110 0x7fa7d41596c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 >> 172.21.15.27:0/457324239 conn(0x7fa7d406c150 msgr2=0x7fa7d40b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 shutdown_connections 2023-07-06T20:15:36.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.713+0000 7fa7dc7ff700 1 -- 172.21.15.27:0/457324239 wait complete. 2023-07-06T20:15:36.717 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2023-07-06T20:15:36.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.936+0000 7f61d51ff700 1 Processor -- start 2023-07-06T20:15:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.937+0000 7f61d51ff700 1 -- start start 2023-07-06T20:15:36.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61d51ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f61d015fca0 0x7f61d00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61d51ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00cf780 con 0x7f61d00b6110 2023-07-06T20:15:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61d51ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00cd490 con 0x7f61d00b5c90 2023-07-06T20:15:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61d51ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00cd770 con 0x7f61d015fca0 2023-07-06T20:15:36.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61cf59e700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f61d00b6110 0x7f61d00cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60680/0 (socket says 172.21.15.27:60680) 2023-07-06T20:15:36.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61cf59e700 1 -- 172.21.15.27:0/461490622 learned_addr learned my addr 172.21.15.27:0/461490622 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:36.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61ced9d700 1 --2- 172.21.15.27:0/461490622 >> v2:172.21.15.153:3300/0 conn(0x7f61d015fca0 0x7f61d00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.938+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532700281 0 0) 0x7f61d00cf780 con 0x7f61d00b6110 2023-07-06T20:15:36.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b80035b0 con 0x7f61d00b6110 2023-07-06T20:15:36.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242453006 0 0) 0x7f61d00cd490 con 0x7f61d00b5c90 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d00cf780 con 0x7f61d00b5c90 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810729607 0 0) 0x7f61b80035b0 con 0x7f61d00b6110 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 >> v2:172.21.15.153:3300/0 conn(0x7f61d015fca0 msgr2=0x7f61d00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 --2- 172.21.15.27:0/461490622 >> v2:172.21.15.153:3300/0 conn(0x7f61d015fca0 0x7f61d00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 >> v1:172.21.15.153:6789/0 conn(0x7f61d00b5c90 legacy=0x7f61d00d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00cda50 con 0x7f61d00b6110 2023-07-06T20:15:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f61c80019f0 con 0x7f61d00b6110 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 >> v1:172.21.15.27:6789/0 conn(0x7f61d00b6110 legacy=0x7f61d00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61b8003b90 0x7f61b8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8004850 0x7f61b8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61b8005200 0x7f61b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00cd770 con 0x7f61b8003b90 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b800c060 con 0x7f61b8004850 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b800c220 con 0x7f61b8005200 2023-07-06T20:15:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61ced9d700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61b8003b90 0x7f61b8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61cf59e700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61b8005200 0x7f61b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.939+0000 7f61ce59c700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8004850 0x7f61b8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61ced9d700 1 -- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61b8005200 msgr2=0x7f61b800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61ced9d700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61b8005200 0x7f61b800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61ced9d700 1 -- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8004850 msgr2=0x7f61b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61ced9d700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8004850 0x7f61b8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61ced9d700 1 -- 172.21.15.27:0/461490622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80035b0 con 0x7f61b8003b90 2023-07-06T20:15:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61cf59e700 1 -- 172.21.15.27:0/461490622 reap_dead start 2023-07-06T20:15:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61ced9d700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61b8003b90 0x7f61b8003f80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009b60 tx=0x7f61b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc97b052233cb6f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000e6c0 con 0x7f61b8003b90 2023-07-06T20:15:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c000e820 con 0x7f61b8003b90 2023-07-06T20:15:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.940+0000 7f61cdd9b700 1 -- 172.21.15.27:0/461490622 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000ebb0 con 0x7f61b8003b90 2023-07-06T20:15:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 -- 172.21.15.27:0/461490622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61b8003b90 msgr2=0x7f61b8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61b8003b90 0x7f61b8003f80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009b60 tx=0x7f61b8005880 comp rx=0 tx=0).stop 2023-07-06T20:15:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 -- 172.21.15.27:0/461490622 shutdown_connections 2023-07-06T20:15:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 --2- 172.21.15.27:0/461490622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61b8003b90 0x7f61b8003f80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 -- 172.21.15.27:0/461490622 >> 172.21.15.27:0/461490622 conn(0x7f61d006c150 msgr2=0x7f61d006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 -- 172.21.15.27:0/461490622 shutdown_connections 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.941+0000 7f61d51ff700 1 -- 172.21.15.27:0/461490622 wait complete. 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.943+0000 7f61d51ff700 1 Processor -- start 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.943+0000 7f61d51ff700 1 -- start start 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61d51ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8003b90 0x7f61d0157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61d51ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 0x7f61d00cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61d51ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d00b3090 0x7f61d00b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61d51ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00c6ec0 con 0x7f61d00ca8f0 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61d51ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00b3a40 con 0x7f61b8003b90 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61d51ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00cb3e0 con 0x7f61d00b3090 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 0x7f61d00cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 0x7f61d00cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42346/0 (socket says 172.21.15.27:42346) 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 -- 172.21.15.27:0/3323921973 learned_addr learned my addr 172.21.15.27:0/3323921973 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:36.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ced9d700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8003b90 0x7f61d0157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61cf59e700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d00b3090 0x7f61d00b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 -- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d00b3090 msgr2=0x7f61d00b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d00b3090 0x7f61d00b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 -- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8003b90 msgr2=0x7f61d0157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8003b90 0x7f61d0157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ce59c700 1 -- 172.21.15.27:0/3323921973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00cb6c0 con 0x7f61d00ca8f0 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.944+0000 7f61ced9d700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8003b90 0x7f61d0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:36.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.945+0000 7f61ce59c700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 0x7f61d00cad60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f61c400e7b0 tx=0x7f61c400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.945+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40052e0 con 0x7f61d00ca8f0 2023-07-06T20:15:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.945+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c4005440 con 0x7f61d00ca8f0 2023-07-06T20:15:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.945+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40038a0 con 0x7f61d00ca8f0 2023-07-06T20:15:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.945+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d00cb9a0 con 0x7f61d00ca8f0 2023-07-06T20:15:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.945+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00c6a70 con 0x7f61d00ca8f0 2023-07-06T20:15:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.946+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d00cbb90 con 0x7f61d00ca8f0 2023-07-06T20:15:36.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.947+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61c4003a00 con 0x7f61d00ca8f0 2023-07-06T20:15:36.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.948+0000 7f61b77fe700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61bc06ad10 0x7f61bc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:36.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.948+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61c4089200 con 0x7f61d00ca8f0 2023-07-06T20:15:36.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.949+0000 7f61ced9d700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61bc06ad10 0x7f61bc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:36.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.949+0000 7f61ced9d700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61bc06ad10 0x7f61bc06d1c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61c00090b0 tx=0x7f61c0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:36.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:36.952+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61c40542b0 con 0x7f61d00ca8f0 2023-07-06T20:15:37.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.158+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61d00c6c90 con 0x7f61d00ca8f0 2023-07-06T20:15:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.160+0000 7f61b77fe700 1 -- 172.21.15.27:0/3323921973 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f61d00c6c90 con 0x7f61d00ca8f0 2023-07-06T20:15:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:37.162 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:18.212485+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61bc06ad10 msgr2=0x7f61bc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61bc06ad10 0x7f61bc06d1c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61c00090b0 tx=0x7f61c0008030 comp rx=0 tx=0).stop 2023-07-06T20:15:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 msgr2=0x7f61d00cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 0x7f61d00cad60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f61c400e7b0 tx=0x7f61c400eb70 comp rx=0 tx=0).stop 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 shutdown_connections 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f61bc06ad10 0x7f61bc06d1c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f61d00b3090 0x7f61d00b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61b8003b90 0x7f61d0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 --2- 172.21.15.27:0/3323921973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61d00ca8f0 0x7f61d00cad60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 >> 172.21.15.27:0/3323921973 conn(0x7f61d006c150 msgr2=0x7f61d00cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.162+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 shutdown_connections 2023-07-06T20:15:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.163+0000 7f61d51ff700 1 -- 172.21.15.27:0/3323921973 wait complete. 2023-07-06T20:15:37.183 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T20:15:37.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.386+0000 7fe1237ff700 1 Processor -- start 2023-07-06T20:15:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.386+0000 7fe1237ff700 1 -- start start 2023-07-06T20:15:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.387+0000 7fe1237ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe11c15fca0 0x7fe11c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.387+0000 7fe1237ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe11c0b5c90 0x7fe11c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.387+0000 7fe1237ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11c0cf780 con 0x7fe11c0b6110 2023-07-06T20:15:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe1237ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0cd490 con 0x7fe11c0b5c90 2023-07-06T20:15:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe1237ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0cd770 con 0x7fe11c15fca0 2023-07-06T20:15:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe11c0b5c90 0x7fe11c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe11c0b5c90 0x7fe11c0d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42352/0 (socket says 172.21.15.27:42352) 2023-07-06T20:15:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 -- 172.21.15.27:0/3065581862 learned_addr learned my addr 172.21.15.27:0/3065581862 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe12159b700 1 --2- 172.21.15.27:0/3065581862 >> v2:172.21.15.153:3300/0 conn(0x7fe11c15fca0 0x7fe11c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 -- 172.21.15.27:0/3065581862 >> v2:172.21.15.153:3300/0 conn(0x7fe11c15fca0 msgr2=0x7fe11c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 --2- 172.21.15.27:0/3065581862 >> v2:172.21.15.153:3300/0 conn(0x7fe11c15fca0 0x7fe11c0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 -- 172.21.15.27:0/3065581862 >> v1:172.21.15.153:6789/0 conn(0x7fe11c0b6110 legacy=0x7fe11c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 -- 172.21.15.27:0/3065581862 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c0cda50 con 0x7fe11c0b5c90 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.388+0000 7fe120d9a700 1 --2- 172.21.15.27:0/3065581862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0d2d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0d0930 tx=0x7fe10c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e385990021ecf584 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe113fff700 1 -- 172.21.15.27:0/3065581862 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00b870 con 0x7fe11c0b5c90 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe113fff700 1 -- 172.21.15.27:0/3065581862 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c00b9d0 con 0x7fe11c0b5c90 2023-07-06T20:15:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe113fff700 1 -- 172.21.15.27:0/3065581862 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00bd60 con 0x7fe11c0b5c90 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 -- 172.21.15.27:0/3065581862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 msgr2=0x7fe11c0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 --2- 172.21.15.27:0/3065581862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0d2d40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0d0930 tx=0x7fe10c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 -- 172.21.15.27:0/3065581862 shutdown_connections 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 --2- 172.21.15.27:0/3065581862 >> v2:172.21.15.153:3300/0 conn(0x7fe11c15fca0 0x7fe11c0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 --2- 172.21.15.27:0/3065581862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0d2d40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 -- 172.21.15.27:0/3065581862 >> 172.21.15.27:0/3065581862 conn(0x7fe11c06c150 msgr2=0x7fe11c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 -- 172.21.15.27:0/3065581862 shutdown_connections 2023-07-06T20:15:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.389+0000 7fe1237ff700 1 -- 172.21.15.27:0/3065581862 wait complete. 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.391+0000 7fe1237ff700 1 Processor -- start 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.391+0000 7fe1237ff700 1 -- start start 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.392+0000 7fe1237ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.392+0000 7fe1237ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11c0b6110 0x7fe11c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.392+0000 7fe1237ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11c15fca0 0x7fe11c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.392+0000 7fe1237ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0c7630 con 0x7fe11c0b5c90 2023-07-06T20:15:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.392+0000 7fe1237ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0c7f90 con 0x7fe11c15fca0 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.392+0000 7fe1237ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0c69b0 con 0x7fe11c0b6110 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42354/0 (socket says 172.21.15.27:42354) 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 -- 172.21.15.27:0/2614948016 learned_addr learned my addr 172.21.15.27:0/2614948016 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe121d9c700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11c15fca0 0x7fe11c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe120d9a700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11c0b6110 0x7fe11c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 -- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11c0b6110 msgr2=0x7fe11c157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11c0b6110 0x7fe11c157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 -- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11c15fca0 msgr2=0x7fe11c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11c15fca0 0x7fe11c0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 -- 172.21.15.27:0/2614948016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009550 con 0x7fe11c0b5c90 2023-07-06T20:15:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.393+0000 7fe12159b700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0c7a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe11800d800 tx=0x7fe11800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.394+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1180126b0 con 0x7fe11c0b5c90 2023-07-06T20:15:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.394+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118012810 con 0x7fe11c0b5c90 2023-07-06T20:15:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.394+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118012bc0 con 0x7fe11c0b5c90 2023-07-06T20:15:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.394+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c0c6c90 con 0x7fe11c0b5c90 2023-07-06T20:15:37.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.394+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c0c7250 con 0x7fe11c0b5c90 2023-07-06T20:15:37.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.395+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c0c7770 con 0x7fe11c0b5c90 2023-07-06T20:15:37.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.396+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe118012d20 con 0x7fe11c0b5c90 2023-07-06T20:15:37.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.397+0000 7fe111ffb700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe10006ace0 0x7fe10006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.397+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1180895c0 con 0x7fe11c0b5c90 2023-07-06T20:15:37.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.397+0000 7fe120d9a700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe10006ace0 0x7fe10006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.398+0000 7fe120d9a700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe10006ace0 0x7fe10006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:37.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.401+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe118058100 con 0x7fe11c0b5c90 2023-07-06T20:15:37.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.650+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe11c0008d0 con 0x7fe11c0b5c90 2023-07-06T20:15:37.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.650+0000 7fe111ffb700 1 -- 172.21.15.27:0/2614948016 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fe118057c90 con 0x7fe11c0b5c90 2023-07-06T20:15:37.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:37.653 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:15:16.714161+0000","modified":"2023-07-06T20:15:20.237603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/827990525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":827990525},{"type":"v1","addr":"172.21.15.27:6835","nonce":827990525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T20:15:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe10006ace0 msgr2=0x7fe10006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe10006ace0 0x7fe10006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe10c000c00 tx=0x7fe10c008030 comp rx=0 tx=0).stop 2023-07-06T20:15:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 msgr2=0x7fe11c0c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0c7a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe11800d800 tx=0x7fe11800db10 comp rx=0 tx=0).stop 2023-07-06T20:15:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 shutdown_connections 2023-07-06T20:15:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe10006ace0 0x7fe10006d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11c15fca0 0x7fe11c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe11c0b5c90 0x7fe11c0c7a50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 --2- 172.21.15.27:0/2614948016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11c0b6110 0x7fe11c157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 >> 172.21.15.27:0/2614948016 conn(0x7fe11c06c150 msgr2=0x7fe11c0d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 shutdown_connections 2023-07-06T20:15:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.653+0000 7fe1237ff700 1 -- 172.21.15.27:0/2614948016 wait complete. 2023-07-06T20:15:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2023-07-06T20:15:37.673 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:15:37.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.875+0000 7fb10adff700 1 Processor -- start 2023-07-06T20:15:37.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.876+0000 7fb10adff700 1 -- start start 2023-07-06T20:15:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10adff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10415fca0 0x7fb1040d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10adff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb1040b5c90 0x7fb1040d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10adff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1040cf780 con 0x7fb1040b6100 2023-07-06T20:15:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10adff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040cd490 con 0x7fb1040b5c90 2023-07-06T20:15:37.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10adff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040cd770 con 0x7fb10415fca0 2023-07-06T20:15:37.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10939c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb1040b6100 0x7fb1040cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60682/0 (socket says 172.21.15.27:60682) 2023-07-06T20:15:37.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb10939c700 1 -- 172.21.15.27:0/2026247474 learned_addr learned my addr 172.21.15.27:0/2026247474 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb108b9b700 1 --2- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10415fca0 0x7fb1040d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.877+0000 7fb103fff700 1 --2- 172.21.15.27:0/2026247474 >> v2:172.21.15.153:3300/0 conn(0x7fb1040b5c90 0x7fb1040d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb1037fe700 1 -- 172.21.15.27:0/2026247474 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906982577 0 0) 0x7fb1040cf780 con 0x7fb1040b6100 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb1037fe700 1 -- 172.21.15.27:0/2026247474 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f40035b0 con 0x7fb1040b6100 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb103fff700 1 -- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10415fca0 msgr2=0x7fb1040d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb103fff700 1 --2- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10415fca0 0x7fb1040d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb103fff700 1 -- 172.21.15.27:0/2026247474 >> v1:172.21.15.27:6789/0 conn(0x7fb1040b6100 legacy=0x7fb1040cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb103fff700 1 -- 172.21.15.27:0/2026247474 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040cda50 con 0x7fb1040b5c90 2023-07-06T20:15:37.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb103fff700 1 --2- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b5c90 0x7fb1040d2d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009840 tx=0x7fb0f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3118a8d1cfae837 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:37.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.878+0000 7fb1037fe700 1 -- 172.21.15.27:0/2026247474 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0011830 con 0x7fb1040b5c90 2023-07-06T20:15:37.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb1037fe700 1 -- 172.21.15.27:0/2026247474 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f0011990 con 0x7fb1040b5c90 2023-07-06T20:15:37.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb1037fe700 1 -- 172.21.15.27:0/2026247474 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0011d20 con 0x7fb1040b5c90 2023-07-06T20:15:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 -- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b5c90 msgr2=0x7fb1040d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 --2- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b5c90 0x7fb1040d2d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009840 tx=0x7fb0f0009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 -- 172.21.15.27:0/2026247474 shutdown_connections 2023-07-06T20:15:37.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 --2- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10415fca0 0x7fb1040d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 --2- 172.21.15.27:0/2026247474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b5c90 0x7fb1040d2d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 -- 172.21.15.27:0/2026247474 >> 172.21.15.27:0/2026247474 conn(0x7fb10406c150 msgr2=0x7fb10406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 -- 172.21.15.27:0/2026247474 shutdown_connections 2023-07-06T20:15:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.879+0000 7fb10adff700 1 -- 172.21.15.27:0/2026247474 wait complete. 2023-07-06T20:15:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.881+0000 7fb10adff700 1 Processor -- start 2023-07-06T20:15:37.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.881+0000 7fb10adff700 1 -- start start 2023-07-06T20:15:37.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10adff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1040b5c90 0x7fb10414fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10adff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b6100 0x7fb1041545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10adff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 0x7fb1041f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10adff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041539b0 con 0x7fb10415fca0 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10adff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041f5850 con 0x7fb1040b6100 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10adff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041f1bd0 con 0x7fb1040b5c90 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.882+0000 7fb10939c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 0x7fb1041f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 0x7fb1041f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42356/0 (socket says 172.21.15.27:42356) 2023-07-06T20:15:37.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb108b9b700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1040b5c90 0x7fb10414fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 -- 172.21.15.27:0/2516477715 learned_addr learned my addr 172.21.15.27:0/2516477715 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb103fff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b6100 0x7fb1041545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 -- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1040b5c90 msgr2=0x7fb10414fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1040b5c90 0x7fb10414fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 -- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b6100 msgr2=0x7fb1041545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b6100 0x7fb1041545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 -- 172.21.15.27:0/2516477715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00094a0 con 0x7fb10415fca0 2023-07-06T20:15:37.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb10939c700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 0x7fb1041f5ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00cbb0 tx=0x7fb0fc00cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:37.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00a9e0 con 0x7fb10415fca0 2023-07-06T20:15:37.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00ab40 con 0x7fb10415fca0 2023-07-06T20:15:37.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.883+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc012400 con 0x7fb10415fca0 2023-07-06T20:15:37.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.884+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1041551b0 con 0x7fb10415fca0 2023-07-06T20:15:37.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.884+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb104155770 con 0x7fb10415fca0 2023-07-06T20:15:37.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.885+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1041f59c0 con 0x7fb10415fca0 2023-07-06T20:15:37.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.886+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0fc01b3b0 con 0x7fb10415fca0 2023-07-06T20:15:37.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.887+0000 7fb1017fa700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb0e806ace0 0x7fb0e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:37.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.887+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0fc089130 con 0x7fb10415fca0 2023-07-06T20:15:37.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.887+0000 7fb108b9b700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb0e806ace0 0x7fb0e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:37.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.888+0000 7fb108b9b700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb0e806ace0 0x7fb0e806d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009b40 tx=0x7fb0f8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:37.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:37.891+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0fc0541f0 con 0x7fb10415fca0 2023-07-06T20:15:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.148+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1040008d0 con 0x7fb10415fca0 2023-07-06T20:15:38.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.149+0000 7fb1017fa700 1 -- 172.21.15.27:0/2516477715 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fb0fc057810 con 0x7fb10415fca0 2023-07-06T20:15:38.151 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:38.151 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T20:15:16.714161+0000","modified":"2023-07-06T20:15:20.237603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/827990525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":827990525},{"type":"v1","addr":"172.21.15.27:6835","nonce":827990525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T20:15:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.151+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb0e806ace0 msgr2=0x7fb0e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.151+0000 7fb10adff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb0e806ace0 0x7fb0e806d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009b40 tx=0x7fb0f8009210 comp rx=0 tx=0).stop 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.151+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 msgr2=0x7fb1041f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.151+0000 7fb10adff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 0x7fb1041f5ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00cbb0 tx=0x7fb0fc00cec0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 shutdown_connections 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb0e806ace0 0x7fb0e806d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb10415fca0 0x7fb1041f5ce0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1040b5c90 0x7fb10414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 --2- 172.21.15.27:0/2516477715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1040b6100 0x7fb1041545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 >> 172.21.15.27:0/2516477715 conn(0x7fb10406c150 msgr2=0x7fb1040d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 shutdown_connections 2023-07-06T20:15:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.152+0000 7fb10adff700 1 -- 172.21.15.27:0/2516477715 wait complete. 2023-07-06T20:15:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2023-07-06T20:15:38.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:38.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.374+0000 7f8f475ff700 1 Processor -- start 2023-07-06T20:15:38.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.374+0000 7f8f475ff700 1 -- start start 2023-07-06T20:15:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f475ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f400d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f475ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8f400b5c90 0x7f8f400d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f475ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f400cf780 con 0x7f8f400b6100 2023-07-06T20:15:38.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f475ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f400cd490 con 0x7f8f400b5c90 2023-07-06T20:15:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f475ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400cd770 con 0x7f8f4015fca0 2023-07-06T20:15:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f44b9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8f400b5c90 0x7f8f400d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f44b9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8f400b5c90 0x7f8f400d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42372/0 (socket says 172.21.15.27:42372) 2023-07-06T20:15:38.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.375+0000 7f8f44b9a700 1 -- 172.21.15.27:0/1162598725 learned_addr learned my addr 172.21.15.27:0/1162598725 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:38.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f4539b700 1 --2- 172.21.15.27:0/1162598725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f400d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f44b9a700 1 -- 172.21.15.27:0/1162598725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 msgr2=0x7f8f400d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:38.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f44b9a700 1 --2- 172.21.15.27:0/1162598725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f400d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f44b9a700 1 -- 172.21.15.27:0/1162598725 >> v1:172.21.15.153:6789/0 conn(0x7f8f400b6100 legacy=0x7f8f400cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f44b9a700 1 -- 172.21.15.27:0/1162598725 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400cda50 con 0x7f8f400b5c90 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f44b9a700 1 --2- 172.21.15.27:0/1162598725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f400d2d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8f300098f0 tx=0x7f8f30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a66bcc6d45d7ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.376+0000 7f8f37fff700 1 -- 172.21.15.27:0/1162598725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30011830 con 0x7f8f400b5c90 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f37fff700 1 -- 172.21.15.27:0/1162598725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30011990 con 0x7f8f400b5c90 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f37fff700 1 -- 172.21.15.27:0/1162598725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30011d20 con 0x7f8f400b5c90 2023-07-06T20:15:38.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 -- 172.21.15.27:0/1162598725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 msgr2=0x7f8f400d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 --2- 172.21.15.27:0/1162598725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f400d2d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8f300098f0 tx=0x7f8f30009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 -- 172.21.15.27:0/1162598725 shutdown_connections 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 --2- 172.21.15.27:0/1162598725 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f400d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 --2- 172.21.15.27:0/1162598725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f400d2d40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 -- 172.21.15.27:0/1162598725 >> 172.21.15.27:0/1162598725 conn(0x7f8f4006c150 msgr2=0x7f8f4006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 -- 172.21.15.27:0/1162598725 shutdown_connections 2023-07-06T20:15:38.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.377+0000 7f8f475ff700 1 -- 172.21.15.27:0/1162598725 wait complete. 2023-07-06T20:15:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.380+0000 7f8f475ff700 1 Processor -- start 2023-07-06T20:15:38.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.380+0000 7f8f475ff700 1 -- start start 2023-07-06T20:15:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f475ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f4015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f475ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f400b6100 0x7f8f4014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f475ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f401f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f475ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4015bf00 con 0x7f8f400b5c90 2023-07-06T20:15:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f475ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400cae00 con 0x7f8f400b6100 2023-07-06T20:15:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f475ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4015c0f0 con 0x7f8f4015fca0 2023-07-06T20:15:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f4539b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f4015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.381+0000 7f8f4539b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f4015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42382/0 (socket says 172.21.15.27:42382) 2023-07-06T20:15:38.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 -- 172.21.15.27:0/4075757707 learned_addr learned my addr 172.21.15.27:0/4075757707 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f45b9c700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f401f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f44b9a700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f400b6100 0x7f8f4014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 -- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 msgr2=0x7f8f401f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f401f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 -- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f400b6100 msgr2=0x7f8f4014c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f400b6100 0x7f8f4014c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 -- 172.21.15.27:0/4075757707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f30009550 con 0x7f8f400b5c90 2023-07-06T20:15:38.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f44b9a700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f400b6100 0x7f8f4014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f4539b700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f4015b9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e870 tx=0x7f8f3c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c009590 con 0x7f8f400b5c90 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c00ee70 con 0x7f8f400b5c90 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.382+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4015c3d0 con 0x7f8f400b5c90 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.383+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00ca50 con 0x7f8f400b5c90 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.383+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f400cb300 con 0x7f8f400b5c90 2023-07-06T20:15:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.384+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4015c560 con 0x7f8f400b5c90 2023-07-06T20:15:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.385+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f3c00cbb0 con 0x7f8f400b5c90 2023-07-06T20:15:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.386+0000 7f8f35ffb700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f2406ace0 0x7f8f2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.386+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f3c0894b0 con 0x7f8f400b5c90 2023-07-06T20:15:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.386+0000 7f8f44b9a700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f2406ace0 0x7f8f2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.387+0000 7f8f44b9a700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f2406ace0 0x7f8f2406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f300098f0 tx=0x7f8f30008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.390+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f3c057fe0 con 0x7f8f400b5c90 2023-07-06T20:15:38.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.596+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f400008d0 con 0x7f8f400b5c90 2023-07-06T20:15:38.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.597+0000 7f8f35ffb700 1 -- 172.21.15.27:0/4075757707 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f8f3c00ce60 con 0x7f8f400b5c90 2023-07-06T20:15:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:18.212485+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:15:15.163204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T20:15:15.707733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f2406ace0 msgr2=0x7f8f2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f2406ace0 0x7f8f2406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f300098f0 tx=0x7f8f30008030 comp rx=0 tx=0).stop 2023-07-06T20:15:38.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 msgr2=0x7f8f4015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f4015b9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e870 tx=0x7f8f3c00ec30 comp rx=0 tx=0).stop 2023-07-06T20:15:38.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 shutdown_connections 2023-07-06T20:15:38.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f2406ace0 0x7f8f2406d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f4015fca0 0x7f8f401f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f400b5c90 0x7f8f4015b9c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 --2- 172.21.15.27:0/4075757707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f400b6100 0x7f8f4014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 >> 172.21.15.27:0/4075757707 conn(0x7f8f4006c150 msgr2=0x7f8f400d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 shutdown_connections 2023-07-06T20:15:38.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.600+0000 7f8f475ff700 1 -- 172.21.15.27:0/4075757707 wait complete. 2023-07-06T20:15:38.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T20:15:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.823+0000 7fb9a1fff700 1 Processor -- start 2023-07-06T20:15:38.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.823+0000 7fb9a1fff700 1 -- start start 2023-07-06T20:15:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb9a1fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb99c15d8b0 0x7fb99c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb9a1fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb99c15dc80 0x7fb99c0d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb9a1fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c06cb60 con 0x7fb99c160640 2023-07-06T20:15:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb9a1fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c161190 con 0x7fb99c15dc80 2023-07-06T20:15:38.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb9a1fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c1613a0 con 0x7fb99c15d8b0 2023-07-06T20:15:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb99c15dc80 0x7fb99c0d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb99c15dc80 0x7fb99c0d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42394/0 (socket says 172.21.15.27:42394) 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 -- 172.21.15.27:0/3859532757 learned_addr learned my addr 172.21.15.27:0/3859532757 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99bfff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb99c160640 0x7fb99c160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60698/0 (socket says 172.21.15.27:60698) 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99b7fe700 1 --2- 172.21.15.27:0/3859532757 >> v2:172.21.15.153:3300/0 conn(0x7fb99c15d8b0 0x7fb99c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 -- 172.21.15.27:0/3859532757 >> v2:172.21.15.153:3300/0 conn(0x7fb99c15d8b0 msgr2=0x7fb99c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 --2- 172.21.15.27:0/3859532757 >> v2:172.21.15.153:3300/0 conn(0x7fb99c15d8b0 0x7fb99c0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 -- 172.21.15.27:0/3859532757 >> v1:172.21.15.27:6789/0 conn(0x7fb99c160640 legacy=0x7fb99c160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99affd700 1 -- 172.21.15.27:0/3859532757 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c06c230 con 0x7fb99c15dc80 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.825+0000 7fb99a7fc700 1 -- 172.21.15.27:0/3859532757 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470241808 0 0) 0x7fb99c06cb60 con 0x7fb99c160640 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.826+0000 7fb99affd700 1 --2- 172.21.15.27:0/3859532757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c15dc80 0x7fb99c0d1080 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9900098f0 tx=0x7fb990009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2575030eb463f769 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.826+0000 7fb99a7fc700 1 -- 172.21.15.27:0/3859532757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990011830 con 0x7fb99c15dc80 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.826+0000 7fb99a7fc700 1 -- 172.21.15.27:0/3859532757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb990011990 con 0x7fb99c15dc80 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.826+0000 7fb99a7fc700 1 -- 172.21.15.27:0/3859532757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990011d20 con 0x7fb99c15dc80 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 -- 172.21.15.27:0/3859532757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c15dc80 msgr2=0x7fb99c0d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/3859532757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c15dc80 0x7fb99c0d1080 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9900098f0 tx=0x7fb990009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:38.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 -- 172.21.15.27:0/3859532757 shutdown_connections 2023-07-06T20:15:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/3859532757 >> v2:172.21.15.153:3300/0 conn(0x7fb99c15d8b0 0x7fb99c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/3859532757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c15dc80 0x7fb99c0d1080 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 -- 172.21.15.27:0/3859532757 >> 172.21.15.27:0/3859532757 conn(0x7fb99c06db20 msgr2=0x7fb99c06df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 -- 172.21.15.27:0/3859532757 shutdown_connections 2023-07-06T20:15:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.827+0000 7fb9a1fff700 1 -- 172.21.15.27:0/3859532757 wait complete. 2023-07-06T20:15:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.828+0000 7fb9a1fff700 1 Processor -- start 2023-07-06T20:15:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.829+0000 7fb9a1fff700 1 -- start start 2023-07-06T20:15:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.829+0000 7fb9a1fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb99c15d8b0 0x7fb99c1e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.829+0000 7fb9a1fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb99c160640 0x7fb99c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb9a1fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 0x7fb99c1f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb9a1fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c1e69e0 con 0x7fb99c0b65e0 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb9a1fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c1e6cc0 con 0x7fb99c160640 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb9a1fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c1e6fa0 con 0x7fb99c15d8b0 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 0x7fb99c1f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 0x7fb99c1f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42402/0 (socket says 172.21.15.27:42402) 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 -- 172.21.15.27:0/2905220240 learned_addr learned my addr 172.21.15.27:0/2905220240 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99b7fe700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb99c15d8b0 0x7fb99c1e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99affd700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb99c160640 0x7fb99c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 -- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb99c15d8b0 msgr2=0x7fb99c1e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb99c15d8b0 0x7fb99c1e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 -- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb99c160640 msgr2=0x7fb99c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb99c160640 0x7fb99c0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.830+0000 7fb99bfff700 1 -- 172.21.15.27:0/2905220240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb990009550 con 0x7fb99c0b65e0 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.831+0000 7fb99bfff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 0x7fb99c1f5670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb99400cba0 tx=0x7fb99400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.831+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99400aaa0 con 0x7fb99c0b65e0 2023-07-06T20:15:38.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.831+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99400ac20 con 0x7fb99c0b65e0 2023-07-06T20:15:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.831+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c1e7280 con 0x7fb99c0b65e0 2023-07-06T20:15:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.831+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940070d0 con 0x7fb99c0b65e0 2023-07-06T20:15:38.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.831+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c0af1c0 con 0x7fb99c0b65e0 2023-07-06T20:15:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.832+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c1e77a0 con 0x7fb99c0b65e0 2023-07-06T20:15:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.833+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb994007230 con 0x7fb99c0b65e0 2023-07-06T20:15:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.834+0000 7fb983fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb98806ace0 0x7fb98806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.834+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb994088f80 con 0x7fb99c0b65e0 2023-07-06T20:15:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.835+0000 7fb99b7fe700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb98806ace0 0x7fb98806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:38.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.836+0000 7fb99b7fe700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb98806ace0 0x7fb98806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009be0 tx=0x7fb98c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:38.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:38.839+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb9940bb310 con 0x7fb99c0b65e0 2023-07-06T20:15:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.094+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb99c160a80 con 0x7fb99c0b65e0 2023-07-06T20:15:39.096 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-07-06T20:15:39.094+0000 7f859e7f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T20:15:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.099+0000 7fb983fff700 1 -- 172.21.15.27:0/2905220240 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb99c1e77a0 con 0x7fb99c0b65e0 2023-07-06T20:15:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb98806ace0 msgr2=0x7fb98806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb98806ace0 0x7fb98806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009be0 tx=0x7fb98c009210 comp rx=0 tx=0).stop 2023-07-06T20:15:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 msgr2=0x7fb99c1f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 0x7fb99c1f5670 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb99400cba0 tx=0x7fb99400ceb0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 shutdown_connections 2023-07-06T20:15:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb98806ace0 0x7fb98806d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb99c0b65e0 0x7fb99c1f5670 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb99c15d8b0 0x7fb99c1e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 --2- 172.21.15.27:0/2905220240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb99c160640 0x7fb99c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 >> 172.21.15.27:0/2905220240 conn(0x7fb99c06db20 msgr2=0x7fb99c0cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 shutdown_connections 2023-07-06T20:15:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.102+0000 7fb9a1fff700 1 -- 172.21.15.27:0/2905220240 wait complete. 2023-07-06T20:15:39.106 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T20:15:39.109 INFO:tasks.ceph.mds.c.smithi027.stderr: -3961> 2023-07-06T20:15:20.096+0000 7f3c53ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T20:15:39.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T20:15:39.166 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-07-06T20:15:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.334+0000 7f53c61ff700 1 Processor -- start 2023-07-06T20:15:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.334+0000 7f53c61ff700 1 -- start start 2023-07-06T20:15:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.335+0000 7f53c61ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f53c015d8b0 0x7f53c00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.335+0000 7f53c61ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c0160d50 0x7f53c00d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.335+0000 7f53c61ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c006cb60 con 0x7f53c015dc80 2023-07-06T20:15:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.335+0000 7f53c61ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c01613a0 con 0x7f53c015d8b0 2023-07-06T20:15:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.335+0000 7f53c61ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c006c230 con 0x7f53c0160d50 2023-07-06T20:15:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bffff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c0160d50 0x7f53c00d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53b7fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f53c015dc80 0x7f53c0160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:48506/0 (socket says 172.21.15.27:48506) 2023-07-06T20:15:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53b7fff700 1 -- 172.21.15.27:0/2616013803 learned_addr learned my addr 172.21.15.27:0/2616013803 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2616013803 >> v2:172.21.15.153:3300/0 conn(0x7f53c015d8b0 0x7f53c00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53beffd700 1 -- 172.21.15.27:0/2616013803 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183681105 0 0) 0x7f53c006cb60 con 0x7f53c015dc80 2023-07-06T20:15:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53beffd700 1 -- 172.21.15.27:0/2616013803 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53ac0035b0 con 0x7f53c015dc80 2023-07-06T20:15:39.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bffff700 1 -- 172.21.15.27:0/2616013803 >> v2:172.21.15.153:3300/0 conn(0x7f53c015d8b0 msgr2=0x7f53c00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bffff700 1 --2- 172.21.15.27:0/2616013803 >> v2:172.21.15.153:3300/0 conn(0x7f53c015d8b0 0x7f53c00d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bffff700 1 -- 172.21.15.27:0/2616013803 >> v1:172.21.15.153:6789/0 conn(0x7f53c015dc80 legacy=0x7f53c0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bffff700 1 -- 172.21.15.27:0/2616013803 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c006c510 con 0x7f53c0160d50 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.336+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2616013803 >> v2:172.21.15.153:3300/0 conn(0x7f53c015d8b0 0x7f53c00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.337+0000 7f53bffff700 1 --2- 172.21.15.27:0/2616013803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c0160d50 0x7f53c00d1080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009dc0 tx=0x7f53b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42bd635634bbfe8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.337+0000 7f53beffd700 1 -- 172.21.15.27:0/2616013803 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800e070 con 0x7f53c0160d50 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.337+0000 7f53beffd700 1 -- 172.21.15.27:0/2616013803 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800bae0 con 0x7f53c0160d50 2023-07-06T20:15:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.337+0000 7f53beffd700 1 -- 172.21.15.27:0/2616013803 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800be90 con 0x7f53c0160d50 2023-07-06T20:15:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.337+0000 7f53c61ff700 1 -- 172.21.15.27:0/2616013803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c0160d50 msgr2=0x7f53c00d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.337+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2616013803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c0160d50 0x7f53c00d1080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009dc0 tx=0x7f53b800b330 comp rx=0 tx=0).stop 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.338+0000 7f53c61ff700 1 -- 172.21.15.27:0/2616013803 shutdown_connections 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.338+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2616013803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c0160d50 0x7f53c00d1080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.338+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2616013803 >> v2:172.21.15.153:3300/0 conn(0x7f53c015d8b0 0x7f53c00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.338+0000 7f53c61ff700 1 -- 172.21.15.27:0/2616013803 >> 172.21.15.27:0/2616013803 conn(0x7f53c006db20 msgr2=0x7f53c006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.338+0000 7f53c61ff700 1 -- 172.21.15.27:0/2616013803 shutdown_connections 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.338+0000 7f53c61ff700 1 -- 172.21.15.27:0/2616013803 wait complete. 2023-07-06T20:15:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.339+0000 7f53c61ff700 1 Processor -- start 2023-07-06T20:15:39.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.340+0000 7f53c61ff700 1 -- start start 2023-07-06T20:15:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.340+0000 7f53c61ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c015d8b0 0x7f53c01f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53c61ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c0160d50 0x7f53c00acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53c61ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 0x7f53c00aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53c61ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00b41d0 con 0x7f53c00ad060 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53c61ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c01f20d0 con 0x7f53c0160d50 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53c61ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00ad880 con 0x7f53c015d8b0 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 0x7f53c00aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 0x7f53c00aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42422/0 (socket says 172.21.15.27:42422) 2023-07-06T20:15:39.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 -- 172.21.15.27:0/2514296697 learned_addr learned my addr 172.21.15.27:0/2514296697 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c015d8b0 0x7f53c01f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53b7fff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c0160d50 0x7f53c00acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 -- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c015d8b0 msgr2=0x7f53c01f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c015d8b0 0x7f53c01f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 -- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c0160d50 msgr2=0x7f53c00acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c0160d50 0x7f53c00acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bffff700 1 -- 172.21.15.27:0/2514296697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00094a0 con 0x7f53c00ad060 2023-07-06T20:15:39.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.341+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c015d8b0 0x7f53c01f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.342+0000 7f53bffff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 0x7f53c00aed10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f53b8014040 tx=0x7f53b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.342+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8004550 con 0x7f53c00ad060 2023-07-06T20:15:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.342+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8020070 con 0x7f53c00ad060 2023-07-06T20:15:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.342+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b8009a70 con 0x7f53c00ad060 2023-07-06T20:15:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.342+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80048f0 con 0x7f53c00ad060 2023-07-06T20:15:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.342+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c00b3b80 con 0x7f53c00ad060 2023-07-06T20:15:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.343+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c00b39e0 con 0x7f53c00ad060 2023-07-06T20:15:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.344+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53b8012430 con 0x7f53c00ad060 2023-07-06T20:15:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.346+0000 7f53bcff9700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53a006ace0 0x7f53a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.346+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f53b8022070 con 0x7f53c00ad060 2023-07-06T20:15:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.346+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53a006ace0 0x7f53a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.347+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53a006ace0 0x7f53a006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009dc0 tx=0x7f53b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:39.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.350+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53b80bd310 con 0x7f53c00ad060 2023-07-06T20:15:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:39.606+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f53c00008d0 con 0x7f53c00ad060 2023-07-06T20:15:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.111+0000 7f53bcff9700 1 -- 172.21.15.27:0/2514296697 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f53c00b39e0 con 0x7f53c00ad060 2023-07-06T20:15:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.113+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53a006ace0 msgr2=0x7f53a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.113+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53a006ace0 0x7f53a006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009dc0 tx=0x7f53b0008040 comp rx=0 tx=0).stop 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.113+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 msgr2=0x7f53c00aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.113+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 0x7f53c00aed10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f53b8014040 tx=0x7f53b8004ad0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 shutdown_connections 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f53a006ace0 0x7f53a006d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00ad060 0x7f53c00aed10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f53c015d8b0 0x7f53c01f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 --2- 172.21.15.27:0/2514296697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c0160d50 0x7f53c00acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 >> 172.21.15.27:0/2514296697 conn(0x7f53c006db20 msgr2=0x7f53c015cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 shutdown_connections 2023-07-06T20:15:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.114+0000 7f53c61ff700 1 -- 172.21.15.27:0/2514296697 wait complete. 2023-07-06T20:15:40.135 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T20:15:40.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.384+0000 7f031a3ff700 1 Processor -- start 2023-07-06T20:15:40.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.384+0000 7f031a3ff700 1 -- start start 2023-07-06T20:15:40.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f031a3ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03140cfe40 0x7f03140cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:40.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f031a3ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0314160cc0 con 0x7f03140b2180 2023-07-06T20:15:40.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f031a3ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0314160f80 con 0x7f03140b8440 2023-07-06T20:15:40.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f031a3ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0314160ae0 con 0x7f03140cfe40 2023-07-06T20:15:40.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f03137fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03140cfe40 0x7f03140cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:40.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f031899c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f03140b2180 0x7f03140d0960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60704/0 (socket says 172.21.15.27:60704) 2023-07-06T20:15:40.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f03137fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03140cfe40 0x7f03140cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42424/0 (socket says 172.21.15.27:42424) 2023-07-06T20:15:40.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.385+0000 7f031899c700 1 -- 172.21.15.27:0/1451953718 learned_addr learned my addr 172.21.15.27:0/1451953718 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:40.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f03137fe700 1 -- 172.21.15.27:0/1451953718 >> v1:172.21.15.153:6789/0 conn(0x7f03140b8440 legacy=0x7f03140cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:40.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f03137fe700 1 -- 172.21.15.27:0/1451953718 >> v1:172.21.15.27:6789/0 conn(0x7f03140b2180 legacy=0x7f03140d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:40.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f03137fe700 1 -- 172.21.15.27:0/1451953718 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140cf0e0 con 0x7f03140cfe40 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f03137fe700 1 --2- 172.21.15.27:0/1451953718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f03140cd530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0308009840 tx=0x7f0308009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56a8f9f984f7d98 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f0312ffd700 1 -- 172.21.15.27:0/1451953718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308004230 con 0x7f03140cfe40 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f0312ffd700 1 -- 172.21.15.27:0/1451953718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0308004390 con 0x7f03140cfe40 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.386+0000 7f0312ffd700 1 -- 172.21.15.27:0/1451953718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03080036a0 con 0x7f03140cfe40 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 -- 172.21.15.27:0/1451953718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 msgr2=0x7f03140cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 --2- 172.21.15.27:0/1451953718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f03140cd530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0308009840 tx=0x7f0308009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:40.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 -- 172.21.15.27:0/1451953718 shutdown_connections 2023-07-06T20:15:40.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 --2- 172.21.15.27:0/1451953718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f03140cd530 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 -- 172.21.15.27:0/1451953718 >> 172.21.15.27:0/1451953718 conn(0x7f031406be10 msgr2=0x7f0314071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:40.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 -- 172.21.15.27:0/1451953718 shutdown_connections 2023-07-06T20:15:40.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.387+0000 7f031a3ff700 1 -- 172.21.15.27:0/1451953718 wait complete. 2023-07-06T20:15:40.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.389+0000 7f031a3ff700 1 Processor -- start 2023-07-06T20:15:40.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.389+0000 7f031a3ff700 1 -- start start 2023-07-06T20:15:40.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031a3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03140b2180 0x7f03141f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:40.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031a3ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03140b8440 0x7f03141f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:40.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031a3ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f0314157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:40.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031a3ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031415dd80 con 0x7f03140cfe40 2023-07-06T20:15:40.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031a3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031415e060 con 0x7f03140b2180 2023-07-06T20:15:40.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031a3ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031415e340 con 0x7f03140b8440 2023-07-06T20:15:40.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031899c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f0314157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:40.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031899c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f0314157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42440/0 (socket says 172.21.15.27:42440) 2023-07-06T20:15:40.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.390+0000 7f031899c700 1 -- 172.21.15.27:0/2985981038 learned_addr learned my addr 172.21.15.27:0/2985981038 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f03137fe700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03140b8440 0x7f03141f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f0313fff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03140b2180 0x7f03141f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031899c700 1 -- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03140b8440 msgr2=0x7f03141f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031899c700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03140b8440 0x7f03141f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031899c700 1 -- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03140b2180 msgr2=0x7f03141f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031899c700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03140b2180 0x7f03141f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:40.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031899c700 1 -- 172.21.15.27:0/2985981038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080094a0 con 0x7f03140cfe40 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f0313fff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03140b2180 0x7f03141f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031899c700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f0314157720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f030c00caf0 tx=0x7f030c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00aad0 con 0x7f03140cfe40 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00ac30 con 0x7f03140cfe40 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c012400 con 0x7f03140cfe40 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031415e620 con 0x7f03140cfe40 2023-07-06T20:15:40.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.391+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03141f5720 con 0x7f03140cfe40 2023-07-06T20:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.393+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031415ec70 con 0x7f03140cfe40 2023-07-06T20:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.394+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f030c01b3b0 con 0x7f03140cfe40 2023-07-06T20:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.394+0000 7f0310ff9700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f030006ace0 0x7f030006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.395+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f030c089040 con 0x7f03140cfe40 2023-07-06T20:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.395+0000 7f0313fff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f030006ace0 0x7f030006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.396+0000 7f0313fff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f030006ace0 0x7f030006d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0304003f40 tx=0x7f0304009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:40.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.399+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f030c0541b0 con 0x7f03140cfe40 2023-07-06T20:15:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:40.608+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03140ce000 con 0x7f03140cfe40 2023-07-06T20:15:41.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.117+0000 7f0310ff9700 1 -- 172.21.15.27:0/2985981038 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f030c0577d0 con 0x7f03140cfe40 2023-07-06T20:15:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f030006ace0 msgr2=0x7f030006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f030006ace0 0x7f030006d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0304003f40 tx=0x7f0304009040 comp rx=0 tx=0).stop 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 msgr2=0x7f0314157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f0314157720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f030c00caf0 tx=0x7f030c00ceb0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 shutdown_connections 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f030006ace0 0x7f030006d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03140cfe40 0x7f0314157720 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03140b2180 0x7f03141f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 --2- 172.21.15.27:0/2985981038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03140b8440 0x7f03141f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.120+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 >> 172.21.15.27:0/2985981038 conn(0x7f031406be10 msgr2=0x7f03140b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.121+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 shutdown_connections 2023-07-06T20:15:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.121+0000 7f031a3ff700 1 -- 172.21.15.27:0/2985981038 wait complete. 2023-07-06T20:15:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-07-06T20:15:41.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T20:15:41.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.354+0000 7f7d80bff700 1 Processor -- start 2023-07-06T20:15:41.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.354+0000 7f7d80bff700 1 -- start start 2023-07-06T20:15:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.355+0000 7f7d80bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7d7c0b8440 0x7f7d7c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.355+0000 7f7d80bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d7c0b2180 0x7f7d7c0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.355+0000 7f7d80bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c160cc0 con 0x7f7d7c0b1c30 2023-07-06T20:15:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.355+0000 7f7d80bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c160ae0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.355+0000 7f7d80bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c160f80 con 0x7f7d7c0b2180 2023-07-06T20:15:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7d7c0b8440 0x7f7d7c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7d7c0b8440 0x7f7d7c0cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42450/0 (socket says 172.21.15.27:42450) 2023-07-06T20:15:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 -- 172.21.15.27:0/4079637638 learned_addr learned my addr 172.21.15.27:0/4079637638 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d79d9b700 1 --2- 172.21.15.27:0/4079637638 >> v2:172.21.15.153:3300/0 conn(0x7f7d7c0b2180 0x7f7d7c0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 -- 172.21.15.27:0/4079637638 >> v2:172.21.15.153:3300/0 conn(0x7f7d7c0b2180 msgr2=0x7f7d7c0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 --2- 172.21.15.27:0/4079637638 >> v2:172.21.15.153:3300/0 conn(0x7f7d7c0b2180 0x7f7d7c0cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 -- 172.21.15.27:0/4079637638 >> v1:172.21.15.27:6789/0 conn(0x7f7d7c0b1c30 legacy=0x7f7d7c0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 -- 172.21.15.27:0/4079637638 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0cf0e0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7a59c700 1 --2- 172.21.15.27:0/4079637638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c0cd530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0b1bc0 tx=0x7f7d6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0b96c39d8354e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.356+0000 7f7d7959a700 1 -- 172.21.15.27:0/4079637638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00b870 con 0x7f7d7c0b8440 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d7959a700 1 -- 172.21.15.27:0/4079637638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c00b9d0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d7959a700 1 -- 172.21.15.27:0/4079637638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00bd60 con 0x7f7d7c0b8440 2023-07-06T20:15:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 -- 172.21.15.27:0/4079637638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 msgr2=0x7f7d7c0cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 --2- 172.21.15.27:0/4079637638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c0cd530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0b1bc0 tx=0x7f7d6c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 -- 172.21.15.27:0/4079637638 shutdown_connections 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 --2- 172.21.15.27:0/4079637638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c0cd530 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 --2- 172.21.15.27:0/4079637638 >> v2:172.21.15.153:3300/0 conn(0x7f7d7c0b2180 0x7f7d7c0cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 -- 172.21.15.27:0/4079637638 >> 172.21.15.27:0/4079637638 conn(0x7f7d7c06be10 msgr2=0x7f7d7c071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 -- 172.21.15.27:0/4079637638 shutdown_connections 2023-07-06T20:15:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.357+0000 7f7d80bff700 1 -- 172.21.15.27:0/4079637638 wait complete. 2023-07-06T20:15:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.360+0000 7f7d80bff700 1 Processor -- start 2023-07-06T20:15:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.360+0000 7f7d80bff700 1 -- start start 2023-07-06T20:15:41.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.361+0000 7f7d80bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d7c0b1c30 0x7f7d7c1e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:41.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.361+0000 7f7d80bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d7c0b2180 0x7f7d7c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:41.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.361+0000 7f7d80bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c1f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:41.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.361+0000 7f7d80bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0cc4d0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.361+0000 7f7d80bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0ca3a0 con 0x7f7d7c0b1c30 2023-07-06T20:15:41.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.361+0000 7f7d80bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0cc720 con 0x7f7d7c0b2180 2023-07-06T20:15:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c1f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d7c0b1c30 0x7f7d7c1e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c1f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42454/0 (socket says 172.21.15.27:42454) 2023-07-06T20:15:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 -- 172.21.15.27:0/1359697238 learned_addr learned my addr 172.21.15.27:0/1359697238 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d79d9b700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d7c0b2180 0x7f7d7c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:41.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 -- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d7c0b2180 msgr2=0x7f7d7c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d7c0b2180 0x7f7d7c0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 -- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d7c0b1c30 msgr2=0x7f7d7c1e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d7c0b1c30 0x7f7d7c1e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 -- 172.21.15.27:0/1359697238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c009550 con 0x7f7d7c0b8440 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d7ad9d700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c1f5ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400cb80 tx=0x7f7d7400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.362+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74003c00 con 0x7f7d7c0b8440 2023-07-06T20:15:41.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.363+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74003d60 con 0x7f7d7c0b8440 2023-07-06T20:15:41.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.363+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0cca00 con 0x7f7d7c0b8440 2023-07-06T20:15:41.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.363+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400aaa0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.363+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c0ccfc0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.364+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c0cd180 con 0x7f7d7c0b8440 2023-07-06T20:15:41.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.365+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d7400ac00 con 0x7f7d7c0b8440 2023-07-06T20:15:41.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.366+0000 7f7d6affd700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7d5c06ace0 0x7f7d5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:41.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.366+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5739+0+0 (secure 0 0 0) 0x7f7d74089220 con 0x7f7d7c0b8440 2023-07-06T20:15:41.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.366+0000 7f7d7a59c700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7d5c06ace0 0x7f7d5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:41.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.367+0000 7f7d7a59c700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7d5c06ace0 0x7f7d5c06d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c00 tx=0x7f7d6c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:41.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.370+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d740543f0 con 0x7f7d7c0b8440 2023-07-06T20:15:41.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:41.585+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7d7c0008d0 con 0x7f7d7c0b8440 2023-07-06T20:15:42.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.123+0000 7f7d6affd700 1 -- 172.21.15.27:0/1359697238 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7d74057a10 con 0x7f7d7c0b8440 2023-07-06T20:15:42.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7d5c06ace0 msgr2=0x7f7d5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7d5c06ace0 0x7f7d5c06d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c00 tx=0x7f7d6c008030 comp rx=0 tx=0).stop 2023-07-06T20:15:42.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 msgr2=0x7f7d7c1f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c1f5ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400cb80 tx=0x7f7d7400ce90 comp rx=0 tx=0).stop 2023-07-06T20:15:42.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 shutdown_connections 2023-07-06T20:15:42.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f7d5c06ace0 0x7f7d5c06d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c0b8440 0x7f7d7c1f5ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d7c0b1c30 0x7f7d7c1e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 --2- 172.21.15.27:0/1359697238 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d7c0b2180 0x7f7d7c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 >> 172.21.15.27:0/1359697238 conn(0x7f7d7c06be10 msgr2=0x7f7d7c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:42.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 shutdown_connections 2023-07-06T20:15:42.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.127+0000 7f7d80bff700 1 -- 172.21.15.27:0/1359697238 wait complete. 2023-07-06T20:15:42.133 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-07-06T20:15:42.151 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T20:15:42.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.357+0000 7fd448bff700 1 Processor -- start 2023-07-06T20:15:42.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.357+0000 7fd448bff700 1 -- start start 2023-07-06T20:15:42.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.358+0000 7fd448bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd44406c330 0x7fd44406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.358+0000 7fd448bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd4440c9480 0x7fd444156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.358+0000 7fd448bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44406cc40 con 0x7fd4440c9010 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.358+0000 7fd448bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4440c9a60 con 0x7fd44406c330 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.358+0000 7fd448bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd444159b70 con 0x7fd4440c9480 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44259c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd44406c330 0x7fd44406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd441d9b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd4440c9010 0x7fd4441565f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50720/0 (socket says 172.21.15.27:50720) 2023-07-06T20:15:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd441d9b700 1 -- 172.21.15.27:0/3565984131 learned_addr learned my addr 172.21.15.27:0/3565984131 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:42.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd442d9d700 1 --2- 172.21.15.27:0/3565984131 >> v2:172.21.15.153:3300/0 conn(0x7fd4440c9480 0x7fd444156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44259c700 1 -- 172.21.15.27:0/3565984131 >> v2:172.21.15.153:3300/0 conn(0x7fd4440c9480 msgr2=0x7fd444156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:42.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44259c700 1 --2- 172.21.15.27:0/3565984131 >> v2:172.21.15.153:3300/0 conn(0x7fd4440c9480 0x7fd444156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44259c700 1 -- 172.21.15.27:0/3565984131 >> v1:172.21.15.27:6789/0 conn(0x7fd4440c9010 legacy=0x7fd4441565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44259c700 1 -- 172.21.15.27:0/3565984131 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444159e50 con 0x7fd44406c330 2023-07-06T20:15:42.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44159a700 1 -- 172.21.15.27:0/3565984131 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241913477 0 0) 0x7fd44406cc40 con 0x7fd4440c9010 2023-07-06T20:15:42.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.359+0000 7fd44259c700 1 --2- 172.21.15.27:0/3565984131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44406c330 0x7fd44406c700 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd44406f6e0 tx=0x7fd434009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f827cc5e1c8646 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:42.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd44159a700 1 -- 172.21.15.27:0/3565984131 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434011830 con 0x7fd44406c330 2023-07-06T20:15:42.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd44159a700 1 -- 172.21.15.27:0/3565984131 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434011990 con 0x7fd44406c330 2023-07-06T20:15:42.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd44159a700 1 -- 172.21.15.27:0/3565984131 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434011d20 con 0x7fd44406c330 2023-07-06T20:15:42.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 -- 172.21.15.27:0/3565984131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44406c330 msgr2=0x7fd44406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 --2- 172.21.15.27:0/3565984131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44406c330 0x7fd44406c700 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd44406f6e0 tx=0x7fd434009b20 comp rx=0 tx=0).stop 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 -- 172.21.15.27:0/3565984131 shutdown_connections 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 --2- 172.21.15.27:0/3565984131 >> v2:172.21.15.153:3300/0 conn(0x7fd4440c9480 0x7fd444156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 --2- 172.21.15.27:0/3565984131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44406c330 0x7fd44406c700 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 -- 172.21.15.27:0/3565984131 >> 172.21.15.27:0/3565984131 conn(0x7fd4440cb440 msgr2=0x7fd4440cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 -- 172.21.15.27:0/3565984131 shutdown_connections 2023-07-06T20:15:42.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.360+0000 7fd448bff700 1 -- 172.21.15.27:0/3565984131 wait complete. 2023-07-06T20:15:42.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.362+0000 7fd448bff700 1 Processor -- start 2023-07-06T20:15:42.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.362+0000 7fd448bff700 1 -- start start 2023-07-06T20:15:42.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.363+0000 7fd448bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44406c330 0x7fd4440b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.363+0000 7fd448bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 0x7fd4440b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.363+0000 7fd448bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4440c9480 0x7fd444155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.363+0000 7fd448bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4440b21c0 con 0x7fd4440c9010 2023-07-06T20:15:42.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.363+0000 7fd448bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd444156690 con 0x7fd4440c9480 2023-07-06T20:15:42.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.363+0000 7fd448bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4440b1890 con 0x7fd44406c330 2023-07-06T20:15:42.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 0x7fd4440b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 0x7fd4440b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57630/0 (socket says 172.21.15.27:57630) 2023-07-06T20:15:42.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 -- 172.21.15.27:0/337037733 learned_addr learned my addr 172.21.15.27:0/337037733 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:42.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd44259c700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44406c330 0x7fd4440b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd442d9d700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4440c9480 0x7fd444155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 -- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44406c330 msgr2=0x7fd4440b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44406c330 0x7fd4440b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 -- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4440c9480 msgr2=0x7fd444155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4440c9480 0x7fd444155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 -- 172.21.15.27:0/337037733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340094a0 con 0x7fd4440c9010 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.364+0000 7fd441d9b700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 0x7fd4440b2650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c520 tx=0x7fd43800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:42.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.365+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438004c80 con 0x7fd4440c9010 2023-07-06T20:15:42.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.365+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438004de0 con 0x7fd4440c9010 2023-07-06T20:15:42.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.365+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4380072d0 con 0x7fd4440c9010 2023-07-06T20:15:42.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.365+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4440b1b70 con 0x7fd4440c9010 2023-07-06T20:15:42.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.365+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4440b46e0 con 0x7fd4440c9010 2023-07-06T20:15:42.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.366+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4440b2330 con 0x7fd4440c9010 2023-07-06T20:15:42.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.367+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd438007430 con 0x7fd4440c9010 2023-07-06T20:15:42.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.368+0000 7fd432ffd700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd42406ace0 0x7fd42406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.368+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd438088ea0 con 0x7fd4440c9010 2023-07-06T20:15:42.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.368+0000 7fd44259c700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd42406ace0 0x7fd42406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.369+0000 7fd44259c700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd42406ace0 0x7fd42406d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4340090b0 tx=0x7fd43401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:42.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.372+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd438054250 con 0x7fd4440c9010 2023-07-06T20:15:42.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.577+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd4440008d0 con 0x7fd4440c9010 2023-07-06T20:15:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.581+0000 7fd432ffd700 1 -- 172.21.15.27:0/337037733 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd438057870 con 0x7fd4440c9010 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd42406ace0 msgr2=0x7fd42406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd42406ace0 0x7fd42406d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4340090b0 tx=0x7fd43401c040 comp rx=0 tx=0).stop 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 msgr2=0x7fd4440b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 0x7fd4440b2650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c520 tx=0x7fd43800c8e0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 shutdown_connections 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fd42406ace0 0x7fd42406d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd4440c9480 0x7fd444155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd44406c330 0x7fd4440b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 --2- 172.21.15.27:0/337037733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4440c9010 0x7fd4440b2650 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 >> 172.21.15.27:0/337037733 conn(0x7fd4440cb440 msgr2=0x7fd44415c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 shutdown_connections 2023-07-06T20:15:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.584+0000 7fd448bff700 1 -- 172.21.15.27:0/337037733 wait complete. 2023-07-06T20:15:42.604 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-07-06T20:15:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.815+0000 7f44a0dff700 1 Processor -- start 2023-07-06T20:15:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.815+0000 7f44a0dff700 1 -- start start 2023-07-06T20:15:42.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.816+0000 7f44a0dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f449c0b6110 0x7f449c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f44a0dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0cf780 con 0x7f449c15fca0 2023-07-06T20:15:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f44a0dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0cd490 con 0x7f449c0b5c90 2023-07-06T20:15:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f44a0dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449c0cd770 con 0x7f449c0b6110 2023-07-06T20:15:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f449ad9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f449c0b6110 0x7f449c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f449a59c700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f449c15fca0 0x7f449c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50730/0 (socket says 172.21.15.27:50730) 2023-07-06T20:15:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f449a59c700 1 -- 172.21.15.27:0/1986999791 learned_addr learned my addr 172.21.15.27:0/1986999791 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f449ad9d700 1 -- 172.21.15.27:0/1986999791 >> v1:172.21.15.153:6789/0 conn(0x7f449c0b5c90 legacy=0x7f449c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f449ad9d700 1 -- 172.21.15.27:0/1986999791 >> v1:172.21.15.27:6789/0 conn(0x7f449c15fca0 legacy=0x7f449c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.817+0000 7f449ad9d700 1 -- 172.21.15.27:0/1986999791 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c0cda50 con 0x7f449c0b6110 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f449ad9d700 1 --2- 172.21.15.27:0/1986999791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b6110 0x7f449c0cfc10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f449400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7855b09aa262e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f4499d9b700 1 -- 172.21.15.27:0/1986999791 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400e070 con 0x7f449c0b6110 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f4499d9b700 1 -- 172.21.15.27:0/1986999791 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449400bb90 con 0x7f449c0b6110 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f4499d9b700 1 -- 172.21.15.27:0/1986999791 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494003a20 con 0x7f449c0b6110 2023-07-06T20:15:42.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f44a0dff700 1 -- 172.21.15.27:0/1986999791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b6110 msgr2=0x7f449c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f44a0dff700 1 --2- 172.21.15.27:0/1986999791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b6110 0x7f449c0cfc10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4494009d90 tx=0x7f449400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f44a0dff700 1 -- 172.21.15.27:0/1986999791 shutdown_connections 2023-07-06T20:15:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f44a0dff700 1 --2- 172.21.15.27:0/1986999791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b6110 0x7f449c0cfc10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f44a0dff700 1 -- 172.21.15.27:0/1986999791 >> 172.21.15.27:0/1986999791 conn(0x7f449c06c150 msgr2=0x7f449c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.818+0000 7f44a0dff700 1 -- 172.21.15.27:0/1986999791 shutdown_connections 2023-07-06T20:15:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.819+0000 7f44a0dff700 1 -- 172.21.15.27:0/1986999791 wait complete. 2023-07-06T20:15:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.820+0000 7f44a0dff700 1 Processor -- start 2023-07-06T20:15:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.820+0000 7f44a0dff700 1 -- start start 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.821+0000 7f44a0dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 0x7f449c14a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.821+0000 7f44a0dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f449c0b6110 0x7f449c0c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.821+0000 7f44a0dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f449c15fca0 0x7f449c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.821+0000 7f44a0dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c14a790 con 0x7f449c0b5c90 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.821+0000 7f44a0dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0c7310 con 0x7f449c15fca0 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.821+0000 7f44a0dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c0c75f0 con 0x7f449c0b6110 2023-07-06T20:15:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 0x7f449c14a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 0x7f449c14a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57646/0 (socket says 172.21.15.27:57646) 2023-07-06T20:15:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 -- 172.21.15.27:0/4178871875 learned_addr learned my addr 172.21.15.27:0/4178871875 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 -- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f449c0b6110 msgr2=0x7f449c0c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f449c0b6110 0x7f449c0c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 -- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f449c15fca0 msgr2=0x7f449c0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f449c15fca0 0x7f449c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 -- 172.21.15.27:0/4178871875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4494009a70 con 0x7f449c0b5c90 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f449a59c700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 0x7f449c14a250 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c490 tx=0x7f448c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:42.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c005d70 con 0x7f449c0b5c90 2023-07-06T20:15:42.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c0070d0 con 0x7f449c0b5c90 2023-07-06T20:15:42.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c007460 con 0x7f449c0b5c90 2023-07-06T20:15:42.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.822+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c0c78d0 con 0x7f449c0b5c90 2023-07-06T20:15:42.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.823+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c0c7e90 con 0x7f449c0b5c90 2023-07-06T20:15:42.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.824+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c0008d0 con 0x7f449c0b5c90 2023-07-06T20:15:42.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.824+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f448c0075c0 con 0x7f449c0b5c90 2023-07-06T20:15:42.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.825+0000 7f4492ffd700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f447c06ace0 0x7f447c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:42.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.826+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f448c088730 con 0x7f449c0b5c90 2023-07-06T20:15:42.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.826+0000 7f4493fff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f447c06ace0 0x7f447c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:42.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.827+0000 7f4493fff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f447c06ace0 0x7f447c06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f449c0b4540 tx=0x7f4484006c50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:42.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:42.830+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f448c053a60 con 0x7f449c0b5c90 2023-07-06T20:15:43.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:43.119+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f449c0c66f0 con 0x7f449c0b5c90 2023-07-06T20:15:44.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.128+0000 7f4492ffd700 1 -- 172.21.15.27:0/4178871875 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v126) v1 ==== 160+0+0 (secure 0 0 0) 0x7f448c057080 con 0x7f449c0b5c90 2023-07-06T20:15:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f447c06ace0 msgr2=0x7f447c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f447c06ace0 0x7f447c06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f449c0b4540 tx=0x7f4484006c50 comp rx=0 tx=0).stop 2023-07-06T20:15:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 msgr2=0x7f449c14a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 0x7f449c14a250 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c490 tx=0x7f448c00c850 comp rx=0 tx=0).stop 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 shutdown_connections 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f447c06ace0 0x7f447c06d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f449c15fca0 0x7f449c0c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449c0b5c90 0x7f449c14a250 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 --2- 172.21.15.27:0/4178871875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f449c0b6110 0x7f449c0c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 >> 172.21.15.27:0/4178871875 conn(0x7f449c06c150 msgr2=0x7f449c15eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 shutdown_connections 2023-07-06T20:15:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:44.131+0000 7f44a0dff700 1 -- 172.21.15.27:0/4178871875 wait complete. 2023-07-06T20:15:44.156 INFO:tasks.cephfs_test_runner:test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) ... ERROR 2023-07-06T20:15:44.156 INFO:tasks.cephfs_test_runner: 2023-07-06T20:15:44.157 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T20:15:44.157 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-07-06T20:15:44.157 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T20:15:44.157 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T20:15:44.157 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-07-06T20:15:44.158 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-07-06T20:15:44.158 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/mount.py", line 1488, in setfattr 2023-07-06T20:15:44.158 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-07-06T20:15:44.158 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-07-06T20:15:44.158 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-07-06T20:15:44.158 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 522, in run 2023-07-06T20:15:44.159 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-07-06T20:15:44.159 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 455, in run 2023-07-06T20:15:44.159 INFO:tasks.cephfs_test_runner: r.wait() 2023-07-06T20:15:44.159 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait 2023-07-06T20:15:44.159 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-06T20:15:44.160 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-06T20:15:44.160 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-06T20:15:44.160 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi027 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-07-06T20:15:44.160 INFO:tasks.cephfs_test_runner: 2023-07-06T20:15:44.160 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T20:15:44.161 INFO:tasks.cephfs_test_runner:Ran 1 test in 58.767s 2023-07-06T20:15:44.161 INFO:tasks.cephfs_test_runner: 2023-07-06T20:15:44.161 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-07-06T20:15:44.161 INFO:tasks.cephfs_test_runner: 2023-07-06T20:15:44.162 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T20:15:44.162 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-07-06T20:15:44.162 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T20:15:44.162 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T20:15:44.162 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-07-06T20:15:44.163 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-07-06T20:15:44.163 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/mount.py", line 1488, in setfattr 2023-07-06T20:15:44.163 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-07-06T20:15:44.163 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-07-06T20:15:44.164 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-07-06T20:15:44.164 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/remote.py", line 522, in run 2023-07-06T20:15:44.164 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-07-06T20:15:44.164 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 455, in run 2023-07-06T20:15:44.164 INFO:tasks.cephfs_test_runner: r.wait() 2023-07-06T20:15:44.164 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait 2023-07-06T20:15:44.165 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-06T20:15:44.165 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-06T20:15:44.165 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-06T20:15:44.165 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi027 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-07-06T20:15:44.165 INFO:tasks.cephfs_test_runner: 2023-07-06T20:15:44.166 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-07-06T20:15:44.229 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fc6b66ab070a493faed86997001cf288 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-07-06T20:15:44.231 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-06T20:15:44.241 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-06T20:15:44.257 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-06T20:15:44.258 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:15:44.258 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:44.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345616k,nr_inodes=4086404,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T20:15:44.313 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-07-06T20:15:44.313 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:44.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345616k,nr_inodes=4086404,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T20:15:44.402 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:15:44.403 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:44.454 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T20:15:44.454 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-07-06T20:15:44.454 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:15:44.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T20:15:44.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.502 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:15:44.552 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T20:15:44.552 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T20:15:44.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T20:15:44.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T20:15:44.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-07-06T20:15:44.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.831 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-07-06T20:15:44.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-07-06T20:15:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2023-07-06T20:15:44.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:15:44.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:44.962 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:15:45.013 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T20:15:45.014 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T20:15:45.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:45.015 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T20:15:45.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:45.090 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T20:15:45.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:45.165 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-07-06T20:15:45.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:45.296 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-07-06T20:15:45.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:15:45.345 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-07-06T20:15:45.419 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-07-06T20:15:45.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:15:45.422 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-06T20:15:45.432 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T20:15:45.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:15:45.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.638+0000 7ff6cc7ff700 1 Processor -- start 2023-07-06T20:15:45.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.639+0000 7ff6cc7ff700 1 -- start start 2023-07-06T20:15:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cc7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cc7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff6c40c9010 0x7ff6c40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cc7ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6c41565f0 0x7ff6c4156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cc7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c406cc40 con 0x7ff6c40c9010 2023-07-06T20:15:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cc7ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c40c9a60 con 0x7ff6c41565f0 2023-07-06T20:15:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cc7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c4159b70 con 0x7ff6c406c330 2023-07-06T20:15:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6c9d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff6c40c9010 0x7ff6c40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6c9d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff6c40c9010 0x7ff6c40c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57658/0 (socket says 172.21.15.27:57658) 2023-07-06T20:15:45.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/1555336460 learned_addr learned my addr 172.21.15.27:0/1555336460 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6cad9c700 1 --2- 172.21.15.27:0/1555336460 >> v2:172.21.15.153:3300/0 conn(0x7ff6c41565f0 0x7ff6c4156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.640+0000 7ff6ca59b700 1 --2- 172.21.15.27:0/1555336460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/1555336460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 msgr2=0x7ff6c406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:45.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9d9a700 1 --2- 172.21.15.27:0/1555336460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/1555336460 >> v2:172.21.15.153:3300/0 conn(0x7ff6c41565f0 msgr2=0x7ff6c4156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9d9a700 1 --2- 172.21.15.27:0/1555336460 >> v2:172.21.15.153:3300/0 conn(0x7ff6c41565f0 0x7ff6c4156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/1555336460 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4159e50 con 0x7ff6c40c9010 2023-07-06T20:15:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6cad9c700 1 --2- 172.21.15.27:0/1555336460 >> v2:172.21.15.153:3300/0 conn(0x7ff6c41565f0 0x7ff6c4156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:45.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9d9a700 1 --2- 172.21.15.27:0/1555336460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c40c9480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406d230 tx=0x7ff6b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a9a9c312f81fc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9599700 1 -- 172.21.15.27:0/1555336460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400b870 con 0x7ff6c40c9010 2023-07-06T20:15:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9599700 1 -- 172.21.15.27:0/1555336460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b400b9d0 con 0x7ff6c40c9010 2023-07-06T20:15:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.641+0000 7ff6c9599700 1 -- 172.21.15.27:0/1555336460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400bd60 con 0x7ff6c40c9010 2023-07-06T20:15:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/1555336460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 msgr2=0x7ff6c40c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:45.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/1555336460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c40c9480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406d230 tx=0x7ff6b4009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/1555336460 shutdown_connections 2023-07-06T20:15:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/1555336460 >> v2:172.21.15.153:3300/0 conn(0x7ff6c41565f0 0x7ff6c4156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/1555336460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/1555336460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c40c9480 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/1555336460 >> 172.21.15.27:0/1555336460 conn(0x7ff6c40cb440 msgr2=0x7ff6c40cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/1555336460 shutdown_connections 2023-07-06T20:15:45.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.642+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/1555336460 wait complete. 2023-07-06T20:15:45.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.644+0000 7ff6cc7ff700 1 Processor -- start 2023-07-06T20:15:45.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.644+0000 7ff6cc7ff700 1 -- start start 2023-07-06T20:15:45.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6cc7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6cc7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c41ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6cc7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c41565f0 0x7ff6c406b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6cc7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40b4560 con 0x7ff6c40c9010 2023-07-06T20:15:45.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6cc7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40c72d0 con 0x7ff6c41565f0 2023-07-06T20:15:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6cc7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c41eda80 con 0x7ff6c406c330 2023-07-06T20:15:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.645+0000 7ff6c9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c41ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c41ed540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57674/0 (socket says 172.21.15.27:57674) 2023-07-06T20:15:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/2533031076 learned_addr learned my addr 172.21.15.27:0/2533031076 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:45.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6ca59b700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6cad9c700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c41565f0 0x7ff6c406b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 msgr2=0x7ff6c40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c41565f0 msgr2=0x7ff6c406b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:45.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c41565f0 0x7ff6c406b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 -- 172.21.15.27:0/2533031076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b4009550 con 0x7ff6c40c9010 2023-07-06T20:15:45.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6c9d9a700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c41ed540 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4000c00 tx=0x7ff6b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:45.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.646+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004310 con 0x7ff6c40c9010 2023-07-06T20:15:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.647+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b401e070 con 0x7ff6c40c9010 2023-07-06T20:15:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.647+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40036a0 con 0x7ff6c40c9010 2023-07-06T20:15:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.647+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c41f0e20 con 0x7ff6c40c9010 2023-07-06T20:15:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.647+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c40c6f60 con 0x7ff6c40c9010 2023-07-06T20:15:45.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.648+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c40c70f0 con 0x7ff6c40c9010 2023-07-06T20:15:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.649+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6b4020030 con 0x7ff6c40c9010 2023-07-06T20:15:45.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.650+0000 7ff6baffd700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff6a806ace0 0x7ff6a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.651+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff6b408a020 con 0x7ff6c40c9010 2023-07-06T20:15:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.651+0000 7ff6ca59b700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff6a806ace0 0x7ff6a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.652+0000 7ff6ca59b700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff6a806ace0 0x7ff6a806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00045f0 tx=0x7ff6c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:45.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.655+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff6b40553e0 con 0x7ff6c40c9010 2023-07-06T20:15:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.861+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6c4159860 con 0x7ff6a806ace0 2023-07-06T20:15:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.864+0000 7ff6baffd700 1 -- 172.21.15.27:0/2533031076 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72569 (secure 0 0 0) 0x7ff6c000b500 con 0x7ff6a806ace0 2023-07-06T20:15:45.867 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff6a806ace0 msgr2=0x7ff6a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff6a806ace0 0x7ff6a806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00045f0 tx=0x7ff6c0008040 comp rx=0 tx=0).stop 2023-07-06T20:15:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 msgr2=0x7ff6c41ed540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c41ed540 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4000c00 tx=0x7ff6b4004750 comp rx=0 tx=0).stop 2023-07-06T20:15:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 shutdown_connections 2023-07-06T20:15:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7ff6a806ace0 0x7ff6a806d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6c41565f0 0x7ff6c406b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6c406c330 0x7ff6c40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 --2- 172.21.15.27:0/2533031076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6c40c9010 0x7ff6c41ed540 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 >> 172.21.15.27:0/2533031076 conn(0x7ff6c40cb440 msgr2=0x7ff6c41554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 shutdown_connections 2023-07-06T20:15:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:45.867+0000 7ff6cc7ff700 1 -- 172.21.15.27:0/2533031076 wait complete. 2023-07-06T20:15:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:15:45.887 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-07-06T20:15:44.368238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":4440,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":4546560,"data_stored":2560771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8401,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.270116"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212319+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:15:39.212319+0000","last_peered":"2023-07-06T20:15:39.212319+0000","last_clean":"2023-07-06T20:15:39.212319+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:39.212319+0000","last_undegraded":"2023-07-06T20:15:39.212319+0000","last_fullsized":"2023-07-06T20:15:39.212319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:58:21.379440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212564+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:15:39.212564+0000","last_peered":"2023-07-06T20:15:39.212564+0000","last_clean":"2023-07-06T20:15:39.212564+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:39.212564+0000","last_undegraded":"2023-07-06T20:15:39.212564+0000","last_fullsized":"2023-07-06T20:15:39.212564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:16:26.940297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.100085+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:15:39.100085+0000","last_peered":"2023-07-06T20:15:39.100085+0000","last_clean":"2023-07-06T20:15:39.100085+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:39.100085+0000","last_undegraded":"2023-07-06T20:15:39.100085+0000","last_fullsized":"2023-07-06T20:15:39.100085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:34.504586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.213073+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:15:39.213073+0000","last_peered":"2023-07-06T20:15:39.213073+0000","last_clean":"2023-07-06T20:15:39.213073+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:39.213073+0000","last_undegraded":"2023-07-06T20:15:39.213073+0000","last_fullsized":"2023-07-06T20:15:39.213073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:34:27.224124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.213095+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:15:39.213095+0000","last_peered":"2023-07-06T20:15:39.213095+0000","last_clean":"2023-07-06T20:15:39.213095+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:39.213095+0000","last_undegraded":"2023-07-06T20:15:39.213095+0000","last_fullsized":"2023-07-06T20:15:39.213095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:49:27.963781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.213153+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:15:39.213153+0000","last_peered":"2023-07-06T20:15:39.213153+0000","last_clean":"2023-07-06T20:15:39.213153+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:15:39.213153+0000","last_undegraded":"2023-07-06T20:15:39.213153+0000","last_fullsized":"2023-07-06T20:15:39.213153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:45.453083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212858+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:15:39.212858+0000","last_peered":"2023-07-06T20:15:39.212858+0000","last_clean":"2023-07-06T20:15:39.212858+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:39.212858+0000","last_undegraded":"2023-07-06T20:15:39.212858+0000","last_fullsized":"2023-07-06T20:15:39.212858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:22:54.669716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212081+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:15:39.212081+0000","last_peered":"2023-07-06T20:15:39.212081+0000","last_clean":"2023-07-06T20:15:39.212081+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:39.212081+0000","last_undegraded":"2023-07-06T20:15:39.212081+0000","last_fullsized":"2023-07-06T20:15:39.212081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:47:31.671470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212662+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:15:39.212662+0000","last_peered":"2023-07-06T20:15:39.212662+0000","last_clean":"2023-07-06T20:15:39.212662+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:39.212662+0000","last_undegraded":"2023-07-06T20:15:39.212662+0000","last_fullsized":"2023-07-06T20:15:39.212662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:16:14.503696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.87},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":170302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.751}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.437}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.316}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.502}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.73499999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.75}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.77900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":1003520,"data_stored":759103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.373}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.626},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.88900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":177357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":11.109999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.324999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.434}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":11.145,"5min":11.145,"15min":11.145},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":11.664,"5min":11.664,"15min":11.664},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":11.208,"5min":11.208,"15min":11.208},"last":0.997},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.997999999999999,"5min":10.997999999999999,"15min":10.997999999999999},"last":0.879},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":11.286,"5min":11.286,"15min":11.286},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":11.055,"5min":11.055,"15min":11.055},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":11.247999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.88800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":170324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.0010000000000003,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.7939999999999996,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":5.1710000000000003,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.624}]},{"osd":5,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":169832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.62}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.747},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:15:45.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:15:46.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.092+0000 7f1d025ff700 1 Processor -- start 2023-07-06T20:15:46.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.092+0000 7f1d025ff700 1 -- start start 2023-07-06T20:15:46.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.093+0000 7f1d025ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1cfc15fca0 0x7f1cfc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.093+0000 7f1d025ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1cfc0b5c90 0x7f1cfc0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.093+0000 7f1d025ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc0cf780 con 0x7f1cfc0b6110 2023-07-06T20:15:46.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.093+0000 7f1d025ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0cd490 con 0x7f1cfc0b5c90 2023-07-06T20:15:46.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.093+0000 7f1d025ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc0cd770 con 0x7f1cfc15fca0 2023-07-06T20:15:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1cfc0b5c90 0x7f1cfc0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1cfc0b5c90 0x7f1cfc0d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57686/0 (socket says 172.21.15.27:57686) 2023-07-06T20:15:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/70670111 learned_addr learned my addr 172.21.15.27:0/70670111 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfbfff700 1 --2- 172.21.15.27:0/70670111 >> v2:172.21.15.153:3300/0 conn(0x7f1cfc15fca0 0x7f1cfc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/70670111 >> v2:172.21.15.153:3300/0 conn(0x7f1cfc15fca0 msgr2=0x7f1cfc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:46.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 --2- 172.21.15.27:0/70670111 >> v2:172.21.15.153:3300/0 conn(0x7f1cfc15fca0 0x7f1cfc0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/70670111 >> v1:172.21.15.153:6789/0 conn(0x7f1cfc0b6110 legacy=0x7f1cfc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/70670111 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc0cda50 con 0x7f1cfc0b5c90 2023-07-06T20:15:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.094+0000 7f1cfb7fe700 1 --2- 172.21.15.27:0/70670111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b5c90 0x7f1cfc0d2d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0d0930 tx=0x7f1cf0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e596bf24824e7e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1cfaffd700 1 -- 172.21.15.27:0/70670111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000b870 con 0x7f1cfc0b5c90 2023-07-06T20:15:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1cfaffd700 1 -- 172.21.15.27:0/70670111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf000b9d0 con 0x7f1cfc0b5c90 2023-07-06T20:15:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1cfaffd700 1 -- 172.21.15.27:0/70670111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000bd60 con 0x7f1cfc0b5c90 2023-07-06T20:15:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 -- 172.21.15.27:0/70670111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b5c90 msgr2=0x7f1cfc0d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 --2- 172.21.15.27:0/70670111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b5c90 0x7f1cfc0d2d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0d0930 tx=0x7f1cf0009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 -- 172.21.15.27:0/70670111 shutdown_connections 2023-07-06T20:15:46.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 --2- 172.21.15.27:0/70670111 >> v2:172.21.15.153:3300/0 conn(0x7f1cfc15fca0 0x7f1cfc0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 --2- 172.21.15.27:0/70670111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b5c90 0x7f1cfc0d2d40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 -- 172.21.15.27:0/70670111 >> 172.21.15.27:0/70670111 conn(0x7f1cfc06c150 msgr2=0x7f1cfc06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 -- 172.21.15.27:0/70670111 shutdown_connections 2023-07-06T20:15:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.095+0000 7f1d025ff700 1 -- 172.21.15.27:0/70670111 wait complete. 2023-07-06T20:15:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.097+0000 7f1d025ff700 1 Processor -- start 2023-07-06T20:15:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.097+0000 7f1d025ff700 1 -- start start 2023-07-06T20:15:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.098+0000 7f1d025ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cfc0b5c90 0x7f1cfc0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.098+0000 7f1d025ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 0x7f1cfc1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.098+0000 7f1d025ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cfc15fca0 0x7f1cfc0b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.098+0000 7f1d025ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc0ccd70 con 0x7f1cfc0b6110 2023-07-06T20:15:46.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.098+0000 7f1d025ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc157bf0 con 0x7f1cfc15fca0 2023-07-06T20:15:46.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.098+0000 7f1d025ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc1f5450 con 0x7f1cfc0b5c90 2023-07-06T20:15:46.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 0x7f1cfc1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 0x7f1cfc1576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57690/0 (socket says 172.21.15.27:57690) 2023-07-06T20:15:46.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/560755686 learned_addr learned my addr 172.21.15.27:0/560755686 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:46.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1d00b9c700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cfc15fca0 0x7f1cfc0b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfbfff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cfc0b5c90 0x7f1cfc0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cfc0b5c90 msgr2=0x7f1cfc0cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cfc0b5c90 0x7f1cfc0cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cfc15fca0 msgr2=0x7f1cfc0b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cfc15fca0 0x7f1cfc0b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 -- 172.21.15.27:0/560755686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0009550 con 0x7f1cfc0b6110 2023-07-06T20:15:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1d00b9c700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cfc15fca0 0x7f1cfc0b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.099+0000 7f1cfb7fe700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 0x7f1cfc1576b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc06d0f0 tx=0x7f1cf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.100+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0004540 con 0x7f1cfc0b6110 2023-07-06T20:15:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.100+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf00036a0 con 0x7f1cfc0b6110 2023-07-06T20:15:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.100+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0003a30 con 0x7f1cfc0b6110 2023-07-06T20:15:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.100+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc1f5730 con 0x7f1cfc0b6110 2023-07-06T20:15:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.100+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc1f5cf0 con 0x7f1cfc0b6110 2023-07-06T20:15:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.101+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc0008d0 con 0x7f1cfc0b6110 2023-07-06T20:15:46.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.102+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cf0021070 con 0x7f1cfc0b6110 2023-07-06T20:15:46.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.103+0000 7f1cf8ff9700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ce806ace0 0x7f1ce806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.103+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1cf008a860 con 0x7f1cfc0b6110 2023-07-06T20:15:46.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.103+0000 7f1cfbfff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ce806ace0 0x7f1ce806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.104+0000 7f1cfbfff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ce806ace0 0x7f1ce806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00b9b0 tx=0x7f1cec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.108+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1cf0055ca0 con 0x7f1cfc0b6110 2023-07-06T20:15:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.309+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cfc06ae40 con 0x7f1ce806ace0 2023-07-06T20:15:46.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.312+0000 7f1cf8ff9700 1 -- 172.21.15.27:0/560755686 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72569 (secure 0 0 0) 0x7f1cec00be80 con 0x7f1ce806ace0 2023-07-06T20:15:46.315 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ce806ace0 msgr2=0x7f1ce806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ce806ace0 0x7f1ce806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00b9b0 tx=0x7f1cec008040 comp rx=0 tx=0).stop 2023-07-06T20:15:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 msgr2=0x7f1cfc1576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 0x7f1cfc1576b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc06d0f0 tx=0x7f1cf0004750 comp rx=0 tx=0).stop 2023-07-06T20:15:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 shutdown_connections 2023-07-06T20:15:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1ce806ace0 0x7f1ce806d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cfc15fca0 0x7f1cfc0b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1cfc0b5c90 0x7f1cfc0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 --2- 172.21.15.27:0/560755686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cfc0b6110 0x7f1cfc1576b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 >> 172.21.15.27:0/560755686 conn(0x7f1cfc06c150 msgr2=0x7f1cfc0b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 shutdown_connections 2023-07-06T20:15:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.315+0000 7f1d025ff700 1 -- 172.21.15.27:0/560755686 wait complete. 2023-07-06T20:15:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:15:46.334 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-07-06T20:15:44.368238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220056,"kb_used_data":4440,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754664,"statfs":{"total":773094113280,"available":772868775936,"internally_reserved":0,"allocated":4546560,"data_stored":2560771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8401,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.270116"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212319+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:15:39.212319+0000","last_peered":"2023-07-06T20:15:39.212319+0000","last_clean":"2023-07-06T20:15:39.212319+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:39.212319+0000","last_undegraded":"2023-07-06T20:15:39.212319+0000","last_fullsized":"2023-07-06T20:15:39.212319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:58:21.379440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212564+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:15:39.212564+0000","last_peered":"2023-07-06T20:15:39.212564+0000","last_clean":"2023-07-06T20:15:39.212564+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:39.212564+0000","last_undegraded":"2023-07-06T20:15:39.212564+0000","last_fullsized":"2023-07-06T20:15:39.212564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:16:26.940297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.100085+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:15:39.100085+0000","last_peered":"2023-07-06T20:15:39.100085+0000","last_clean":"2023-07-06T20:15:39.100085+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:39.100085+0000","last_undegraded":"2023-07-06T20:15:39.100085+0000","last_fullsized":"2023-07-06T20:15:39.100085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:34.504586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.213073+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:15:39.213073+0000","last_peered":"2023-07-06T20:15:39.213073+0000","last_clean":"2023-07-06T20:15:39.213073+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:39.213073+0000","last_undegraded":"2023-07-06T20:15:39.213073+0000","last_fullsized":"2023-07-06T20:15:39.213073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:34:27.224124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.213095+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:15:39.213095+0000","last_peered":"2023-07-06T20:15:39.213095+0000","last_clean":"2023-07-06T20:15:39.213095+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:39.213095+0000","last_undegraded":"2023-07-06T20:15:39.213095+0000","last_fullsized":"2023-07-06T20:15:39.213095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:49:27.963781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.213153+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:15:39.213153+0000","last_peered":"2023-07-06T20:15:39.213153+0000","last_clean":"2023-07-06T20:15:39.213153+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:15:39.213153+0000","last_undegraded":"2023-07-06T20:15:39.213153+0000","last_fullsized":"2023-07-06T20:15:39.213153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:45.453083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212858+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:15:39.212858+0000","last_peered":"2023-07-06T20:15:39.212858+0000","last_clean":"2023-07-06T20:15:39.212858+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:39.212858+0000","last_undegraded":"2023-07-06T20:15:39.212858+0000","last_fullsized":"2023-07-06T20:15:39.212858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:22:54.669716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212081+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:15:39.212081+0000","last_peered":"2023-07-06T20:15:39.212081+0000","last_clean":"2023-07-06T20:15:39.212081+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:39.212081+0000","last_undegraded":"2023-07-06T20:15:39.212081+0000","last_fullsized":"2023-07-06T20:15:39.212081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:47:31.671470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T20:15:39.212662+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:15:39.212662+0000","last_peered":"2023-07-06T20:15:39.212662+0000","last_clean":"2023-07-06T20:15:39.212662+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:39.212662+0000","last_undegraded":"2023-07-06T20:15:39.212662+0000","last_fullsized":"2023-07-06T20:15:39.212662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:16:14.503696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.87},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":170302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.751}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.437}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.316}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.502}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.73499999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.75}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.77900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":1003520,"data_stored":759103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.373}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.626},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":169790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.88900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":177357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":11.109999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.324999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.434}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":11.145,"5min":11.145,"15min":11.145},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":11.664,"5min":11.664,"15min":11.664},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":11.208,"5min":11.208,"15min":11.208},"last":0.997},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.997999999999999,"5min":10.997999999999999,"15min":10.997999999999999},"last":0.879},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":11.286,"5min":11.286,"15min":11.286},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":11.055,"5min":11.055,"15min":11.055},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":11.247999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.88800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":170324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.0010000000000003,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.7939999999999996,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":5.1710000000000003,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.624}]},{"osd":5,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":169832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.62}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.747},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:15:46.337 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T20:15:46.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:15:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.539+0000 7f74badff700 1 Processor -- start 2023-07-06T20:15:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.539+0000 7f74badff700 1 -- start start 2023-07-06T20:15:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74badff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74badff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74b40b6100 0x7f74b40cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74badff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40cf780 con 0x7f74b40b5c90 2023-07-06T20:15:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74badff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b40cd490 con 0x7f74b40b6100 2023-07-06T20:15:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74badff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40cd770 con 0x7f74b415fca0 2023-07-06T20:15:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74b939c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74b40b6100 0x7f74b40cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74b8b9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74b3fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f74b40b5c90 0x7f74b40d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:44910/0 (socket says 172.21.15.27:44910) 2023-07-06T20:15:46.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74b8b9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b40d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:55342/0 (socket says 172.21.15.27:55342) 2023-07-06T20:15:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74b3fff700 1 -- 172.21.15.27:0/3973600389 learned_addr learned my addr 172.21.15.27:0/3973600389 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:46.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.541+0000 7f74b939c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74b40b6100 0x7f74b40cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57694/0 (socket says 172.21.15.27:57694) 2023-07-06T20:15:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b37fe700 1 -- 172.21.15.27:0/3973600389 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502901240 0 0) 0x7f74b40cf780 con 0x7f74b40b5c90 2023-07-06T20:15:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b8b9b700 1 -- 172.21.15.27:0/3973600389 >> v2:172.21.15.27:3300/0 conn(0x7f74b40b6100 msgr2=0x7f74b40cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b8b9b700 1 --2- 172.21.15.27:0/3973600389 >> v2:172.21.15.27:3300/0 conn(0x7f74b40b6100 0x7f74b40cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b8b9b700 1 -- 172.21.15.27:0/3973600389 >> v1:172.21.15.153:6789/0 conn(0x7f74b40b5c90 legacy=0x7f74b40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b8b9b700 1 -- 172.21.15.27:0/3973600389 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40cda50 con 0x7f74b415fca0 2023-07-06T20:15:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b8b9b700 1 --2- 172.21.15.27:0/3973600389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b40d1420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009840 tx=0x7f74a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f937733f6a16941 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b37fe700 1 -- 172.21.15.27:0/3973600389 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8011830 con 0x7f74b415fca0 2023-07-06T20:15:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b37fe700 1 -- 172.21.15.27:0/3973600389 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8011990 con 0x7f74b415fca0 2023-07-06T20:15:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.542+0000 7f74b37fe700 1 -- 172.21.15.27:0/3973600389 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8011d20 con 0x7f74b415fca0 2023-07-06T20:15:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 -- 172.21.15.27:0/3973600389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 msgr2=0x7f74b40d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 --2- 172.21.15.27:0/3973600389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b40d1420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009840 tx=0x7f74a8009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 -- 172.21.15.27:0/3973600389 shutdown_connections 2023-07-06T20:15:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 --2- 172.21.15.27:0/3973600389 >> v2:172.21.15.27:3300/0 conn(0x7f74b40b6100 0x7f74b40cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 --2- 172.21.15.27:0/3973600389 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b40d1420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 -- 172.21.15.27:0/3973600389 >> 172.21.15.27:0/3973600389 conn(0x7f74b406c150 msgr2=0x7f74b406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 -- 172.21.15.27:0/3973600389 shutdown_connections 2023-07-06T20:15:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.543+0000 7f74badff700 1 -- 172.21.15.27:0/3973600389 wait complete. 2023-07-06T20:15:46.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.545+0000 7f74badff700 1 Processor -- start 2023-07-06T20:15:46.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.545+0000 7f74badff700 1 -- start start 2023-07-06T20:15:46.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74badff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 0x7f74b4153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74badff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b40b6100 0x7f74b4156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74badff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74badff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4155920 con 0x7f74b40b5c90 2023-07-06T20:15:46.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74badff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4154100 con 0x7f74b40b6100 2023-07-06T20:15:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74badff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4155e80 con 0x7f74b415fca0 2023-07-06T20:15:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74b8b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 0x7f74b4153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74b8b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 0x7f74b4153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57702/0 (socket says 172.21.15.27:57702) 2023-07-06T20:15:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.546+0000 7f74b8b9b700 1 -- 172.21.15.27:0/2394378153 learned_addr learned my addr 172.21.15.27:0/2394378153 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:46.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b3fff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b40b6100 0x7f74b4156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b939c700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b8b9b700 1 -- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 msgr2=0x7f74b415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b8b9b700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b8b9b700 1 -- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b40b6100 msgr2=0x7f74b4156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b8b9b700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b40b6100 0x7f74b4156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b8b9b700 1 -- 172.21.15.27:0/2394378153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80094a0 con 0x7f74b40b5c90 2023-07-06T20:15:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b8b9b700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 0x7f74b4153bc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009fd0 tx=0x7f74a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80038a0 con 0x7f74b40b5c90 2023-07-06T20:15:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a801f070 con 0x7f74b40b5c90 2023-07-06T20:15:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8003c40 con 0x7f74b40b5c90 2023-07-06T20:15:46.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4154ae0 con 0x7f74b40b5c90 2023-07-06T20:15:46.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.547+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b40b41f0 con 0x7f74b40b5c90 2023-07-06T20:15:46.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.548+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b40b4400 con 0x7f74b40b5c90 2023-07-06T20:15:46.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.549+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f74a8026480 con 0x7f74b40b5c90 2023-07-06T20:15:46.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.550+0000 7f74b17fa700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f749806ace0 0x7f749806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.551+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f74a8021070 con 0x7f74b40b5c90 2023-07-06T20:15:46.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.551+0000 7f74b3fff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f749806ace0 0x7f749806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.552+0000 7f74b3fff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f749806ace0 0x7f749806d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74b41f5ab0 tx=0x7f74a0007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.554+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f74a8055dc0 con 0x7f74b40b5c90 2023-07-06T20:15:46.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.758+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74b406ae40 con 0x7f749806ace0 2023-07-06T20:15:46.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.761+0000 7f74b17fa700 1 -- 172.21.15.27:0/2394378153 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72664 (secure 0 0 0) 0x7f74a00012b0 con 0x7f749806ace0 2023-07-06T20:15:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.763+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f749806ace0 msgr2=0x7f749806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.763+0000 7f74badff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f749806ace0 0x7f749806d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74b41f5ab0 tx=0x7f74a0007be0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 msgr2=0x7f74b4153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 0x7f74b4153bc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009fd0 tx=0x7f74a8004750 comp rx=0 tx=0).stop 2023-07-06T20:15:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 shutdown_connections 2023-07-06T20:15:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f749806ace0 0x7f749806d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f74b415fca0 0x7f74b415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74b40b5c90 0x7f74b4153bc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 --2- 172.21.15.27:0/2394378153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74b40b6100 0x7f74b4156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 >> 172.21.15.27:0/2394378153 conn(0x7f74b406c150 msgr2=0x7f74b40b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 shutdown_connections 2023-07-06T20:15:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.764+0000 7f74badff700 1 -- 172.21.15.27:0/2394378153 wait complete. 2023-07-06T20:15:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:15:46.783 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-07-06T20:15:46.368640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220420,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754300,"statfs":{"total":773094113280,"available":772868403200,"internally_reserved":0,"allocated":4919296,"data_stored":2739517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8401,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.269169"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630997+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:15:42.630997+0000","last_peered":"2023-07-06T20:15:42.630997+0000","last_clean":"2023-07-06T20:15:42.630997+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:42.630997+0000","last_undegraded":"2023-07-06T20:15:42.630997+0000","last_fullsized":"2023-07-06T20:15:42.630997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:44:21.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.632122+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:15:42.632122+0000","last_peered":"2023-07-06T20:15:42.632122+0000","last_clean":"2023-07-06T20:15:42.632122+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:42.632122+0000","last_undegraded":"2023-07-06T20:15:42.632122+0000","last_fullsized":"2023-07-06T20:15:42.632122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:16:31.488104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630991+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:15:42.630991+0000","last_peered":"2023-07-06T20:15:42.630991+0000","last_clean":"2023-07-06T20:15:42.630991+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:42.630991+0000","last_undegraded":"2023-07-06T20:15:42.630991+0000","last_fullsized":"2023-07-06T20:15:42.630991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:28:43.517463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630297+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:15:42.630297+0000","last_peered":"2023-07-06T20:15:42.630297+0000","last_clean":"2023-07-06T20:15:42.630297+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:42.630297+0000","last_undegraded":"2023-07-06T20:15:42.630297+0000","last_fullsized":"2023-07-06T20:15:42.630297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:40:46.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.631396+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:15:42.631396+0000","last_peered":"2023-07-06T20:15:42.631396+0000","last_clean":"2023-07-06T20:15:42.631396+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:42.631396+0000","last_undegraded":"2023-07-06T20:15:42.631396+0000","last_fullsized":"2023-07-06T20:15:42.631396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:49.298285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.631433+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:15:42.631433+0000","last_peered":"2023-07-06T20:15:42.631433+0000","last_clean":"2023-07-06T20:15:42.631433+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:15:42.631433+0000","last_undegraded":"2023-07-06T20:15:42.631433+0000","last_fullsized":"2023-07-06T20:15:42.631433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:43.584582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630552+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:15:42.630552+0000","last_peered":"2023-07-06T20:15:42.630552+0000","last_clean":"2023-07-06T20:15:42.630552+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:42.630552+0000","last_undegraded":"2023-07-06T20:15:42.630552+0000","last_fullsized":"2023-07-06T20:15:42.630552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:16:31.488104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.632088+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:15:42.632088+0000","last_peered":"2023-07-06T20:15:42.632088+0000","last_clean":"2023-07-06T20:15:42.632088+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:42.632088+0000","last_undegraded":"2023-07-06T20:15:42.632088+0000","last_fullsized":"2023-07-06T20:15:42.632088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:44:04.052137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.357268+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:15:42.357268+0000","last_peered":"2023-07-06T20:15:42.357268+0000","last_clean":"2023-07-06T20:15:42.357268+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:42.357268+0000","last_undegraded":"2023-07-06T20:15:42.357268+0000","last_fullsized":"2023-07-06T20:15:42.357268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:17:04.184086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":1060864,"data_stored":785343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.60699999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.7879999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":3.4830000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.752}]},{"osd":5,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.878}]},{"osd":7,"last update":"Thu Jul 6 20:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.62}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":1056768,"data_stored":784324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jul 6 20:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.91100000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.80400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":11.109999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.324999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":11.145,"5min":11.145,"15min":11.145},"last":0.432}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":11.664,"5min":11.664,"15min":11.664},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":11.208,"5min":11.208,"15min":11.208},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.95199999999999996}]},{"osd":5,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.997999999999999,"5min":10.997999999999999,"15min":10.997999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":11.286,"5min":11.286,"15min":11.286},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":11.055,"5min":11.055,"15min":11.055},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":11.247999999999999,"5min":11.247999999999999,"15min":11.247999999999999},"last":1.024},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.879}]},{"osd":1,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.0010000000000003,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.7939999999999996,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.99199999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":5.1710000000000003,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":1.038}]},{"osd":7,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91700000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.99399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:15:46.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:15:46.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.988+0000 7f6330fff700 1 Processor -- start 2023-07-06T20:15:46.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.988+0000 7f6330fff700 1 -- start start 2023-07-06T20:15:46.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f6330fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f632c15fca0 0x7f632c0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f6330fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f6330fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f632c0b60e0 0x7f632c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:46.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f6330fff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0cf780 con 0x7f632c15fca0 2023-07-06T20:15:46.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f6330fff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0cd490 con 0x7f632c0b60e0 2023-07-06T20:15:46.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f6330fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0cd770 con 0x7f632c0b5c90 2023-07-06T20:15:46.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.990+0000 7f632a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f632c15fca0 0x7f632c0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f632c15fca0 0x7f632c0d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57712/0 (socket says 172.21.15.27:57712) 2023-07-06T20:15:46.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 -- 172.21.15.27:0/2894780655 learned_addr learned my addr 172.21.15.27:0/2894780655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:46.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f6323fff700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632ad9d700 1 --2- 172.21.15.27:0/2894780655 >> v2:172.21.15.153:3300/0 conn(0x7f632c0b60e0 0x7f632c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 -- 172.21.15.27:0/2894780655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 msgr2=0x7f632c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 -- 172.21.15.27:0/2894780655 >> v2:172.21.15.153:3300/0 conn(0x7f632c0b60e0 msgr2=0x7f632c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 --2- 172.21.15.27:0/2894780655 >> v2:172.21.15.153:3300/0 conn(0x7f632c0b60e0 0x7f632c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 -- 172.21.15.27:0/2894780655 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0cda50 con 0x7f632c15fca0 2023-07-06T20:15:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632ad9d700 1 --2- 172.21.15.27:0/2894780655 >> v2:172.21.15.153:3300/0 conn(0x7f632c0b60e0 0x7f632c0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f6323fff700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.991+0000 7f632a59c700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c15fca0 0x7f632c0d1420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f632c0b8f80 tx=0x7f631c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beabb00078b3f26a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6329d9b700 1 -- 172.21.15.27:0/2894780655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00b870 con 0x7f632c15fca0 2023-07-06T20:15:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6329d9b700 1 -- 172.21.15.27:0/2894780655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631c00b9d0 con 0x7f632c15fca0 2023-07-06T20:15:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6329d9b700 1 -- 172.21.15.27:0/2894780655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00bd60 con 0x7f632c15fca0 2023-07-06T20:15:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 -- 172.21.15.27:0/2894780655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c15fca0 msgr2=0x7f632c0d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c15fca0 0x7f632c0d1420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f632c0b8f80 tx=0x7f631c009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 -- 172.21.15.27:0/2894780655 shutdown_connections 2023-07-06T20:15:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 --2- 172.21.15.27:0/2894780655 >> v2:172.21.15.153:3300/0 conn(0x7f632c0b60e0 0x7f632c0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c15fca0 0x7f632c0d1420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 --2- 172.21.15.27:0/2894780655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 -- 172.21.15.27:0/2894780655 >> 172.21.15.27:0/2894780655 conn(0x7f632c06c150 msgr2=0x7f632c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:46.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 -- 172.21.15.27:0/2894780655 shutdown_connections 2023-07-06T20:15:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.992+0000 7f6330fff700 1 -- 172.21.15.27:0/2894780655 wait complete. 2023-07-06T20:15:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.994+0000 7f6330fff700 1 Processor -- start 2023-07-06T20:15:47.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.994+0000 7f6330fff700 1 -- start start 2023-07-06T20:15:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6330fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:47.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6330fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 0x7f632c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6330fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f632c15fca0 0x7f632c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6330fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15b730 con 0x7f632c0b60e0 2023-07-06T20:15:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6330fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15b980 con 0x7f632c15fca0 2023-07-06T20:15:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6330fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15bc60 con 0x7f632c0b5c90 2023-07-06T20:15:47.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6323fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 0x7f632c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:47.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.995+0000 7f6323fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 0x7f632c0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57718/0 (socket says 172.21.15.27:57718) 2023-07-06T20:15:47.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 -- 172.21.15.27:0/367529999 learned_addr learned my addr 172.21.15.27:0/367529999 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:47.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f632a59c700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:47.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 -- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 msgr2=0x7f632c153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:47.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f632ad9d700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f632c15fca0 0x7f632c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:47.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 -- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f632c15fca0 msgr2=0x7f632c15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:47.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f632c15fca0 0x7f632c15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 -- 172.21.15.27:0/367529999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c009550 con 0x7f632c0b60e0 2023-07-06T20:15:47.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f632a59c700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:47.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f632ad9d700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f632c15fca0 0x7f632c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:47.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6323fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 0x7f632c0b28c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f631400d800 tx=0x7f631400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:47.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314004030 con 0x7f632c0b60e0 2023-07-06T20:15:47.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314004190 con 0x7f632c0b60e0 2023-07-06T20:15:47.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314004520 con 0x7f632c0b60e0 2023-07-06T20:15:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.996+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c15bf40 con 0x7f632c0b60e0 2023-07-06T20:15:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.997+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c154170 con 0x7f632c0b60e0 2023-07-06T20:15:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.998+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c0008d0 con 0x7f632c0b60e0 2023-07-06T20:15:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.999+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63140036a0 con 0x7f632c0b60e0 2023-07-06T20:15:47.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:46.999+0000 7f6322ffd700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f630c06ace0 0x7f630c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.000+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6314089680 con 0x7f632c0b60e0 2023-07-06T20:15:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.000+0000 7f632a59c700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f630c06ace0 0x7f630c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.001+0000 7f632a59c700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f630c06ace0 0x7f630c06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f632c0d0430 tx=0x7f631c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:47.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.004+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63140584d0 con 0x7f632c0b60e0 2023-07-06T20:15:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.209+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f632c0b8d40 con 0x7f632c0b60e0 2023-07-06T20:15:47.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.210+0000 7f6322ffd700 1 -- 172.21.15.27:0/367529999 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+14742 (secure 0 0 0) 0x7f6314058060 con 0x7f632c0b60e0 2023-07-06T20:15:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:47.212 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":40,"fsid":"c2a9da8e-f30b-43f7-b235-3303ae838054","created":"2023-07-06T20:13:31.024459+0000","modified":"2023-07-06T20:15:42.116746+0000","last_up_change":"2023-07-06T20:13:43.489960+0000","last_in_change":"2023-07-06T20:13:34.501804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:44.411793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T20:13:51.018054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfbccfdc-2546-4e99-b098-57a4455400fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6801","nonce":323630915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6803","nonce":323630915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6807","nonce":323630915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":323630915},{"type":"v1","addr":"172.21.15.27:6805","nonce":323630915}]},"public_addr":"172.21.15.27:6801/323630915","cluster_addr":"172.21.15.27:6803/323630915","heartbeat_back_addr":"172.21.15.27:6807/323630915","heartbeat_front_addr":"172.21.15.27:6805/323630915","state":["exists","up"]},{"osd":1,"uuid":"42558411-4aac-4fbd-8e61-4349de6d8b6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6825","nonce":3450674250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6827","nonce":3450674250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6831","nonce":3450674250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3450674250},{"type":"v1","addr":"172.21.15.27:6829","nonce":3450674250}]},"public_addr":"172.21.15.27:6825/3450674250","cluster_addr":"172.21.15.27:6827/3450674250","heartbeat_back_addr":"172.21.15.27:6831/3450674250","heartbeat_front_addr":"172.21.15.27:6829/3450674250","state":["exists","up"]},{"osd":2,"uuid":"dee0e9d7-ef17-4054-a52f-2ff3bc8534e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6810","nonce":380505300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6814","nonce":380505300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6822","nonce":380505300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":380505300},{"type":"v1","addr":"172.21.15.27:6818","nonce":380505300}]},"public_addr":"172.21.15.27:6810/380505300","cluster_addr":"172.21.15.27:6814/380505300","heartbeat_back_addr":"172.21.15.27:6822/380505300","heartbeat_front_addr":"172.21.15.27:6818/380505300","state":["exists","up"]},{"osd":3,"uuid":"86e15cd3-5bb1-4ce2-ad04-f0235fd0782e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6809","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6811","nonce":2399591703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6813","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6815","nonce":2399591703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6823","nonce":2399591703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6817","nonce":2399591703},{"type":"v1","addr":"172.21.15.27:6819","nonce":2399591703}]},"public_addr":"172.21.15.27:6811/2399591703","cluster_addr":"172.21.15.27:6815/2399591703","heartbeat_back_addr":"172.21.15.27:6823/2399591703","heartbeat_front_addr":"172.21.15.27:6819/2399591703","state":["exists","up"]},{"osd":4,"uuid":"8176a7ab-1538-4693-a632-a07c7bcd3ad0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6801","nonce":2660030028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6803","nonce":2660030028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6807","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6809","nonce":2660030028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2660030028},{"type":"v1","addr":"172.21.15.153:6805","nonce":2660030028}]},"public_addr":"172.21.15.153:6801/2660030028","cluster_addr":"172.21.15.153:6803/2660030028","heartbeat_back_addr":"172.21.15.153:6809/2660030028","heartbeat_front_addr":"172.21.15.153:6805/2660030028","state":["exists","up"]},{"osd":5,"uuid":"3e5a4020-5fa4-4e47-80bd-b03c0c0af152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6808","nonce":1975714118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6811","nonce":1975714118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6815","nonce":1975714118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1975714118},{"type":"v1","addr":"172.21.15.153:6813","nonce":1975714118}]},"public_addr":"172.21.15.153:6808/1975714118","cluster_addr":"172.21.15.153:6811/1975714118","heartbeat_back_addr":"172.21.15.153:6815/1975714118","heartbeat_front_addr":"172.21.15.153:6813/1975714118","state":["exists","up"]},{"osd":6,"uuid":"ad6e9be7-c296-45db-a17f-ebb2600dc3f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6825","nonce":2351180975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6827","nonce":2351180975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6831","nonce":2351180975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":2351180975},{"type":"v1","addr":"172.21.15.153:6829","nonce":2351180975}]},"public_addr":"172.21.15.153:6825/2351180975","cluster_addr":"172.21.15.153:6827/2351180975","heartbeat_back_addr":"172.21.15.153:6831/2351180975","heartbeat_front_addr":"172.21.15.153:6829/2351180975","state":["exists","up"]},{"osd":7,"uuid":"4e6b4d72-73d6-49cf-9d5f-bff9a56eb71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6817","nonce":469437508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6819","nonce":469437508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6823","nonce":469437508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":469437508},{"type":"v1","addr":"172.21.15.153:6821","nonce":469437508}]},"public_addr":"172.21.15.153:6817/469437508","cluster_addr":"172.21.15.153:6819/469437508","heartbeat_back_addr":"172.21.15.153:6823/469437508","heartbeat_front_addr":"172.21.15.153:6821/469437508","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.552141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.876392+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.918679+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.751356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.863593+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:39.737542+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.154144+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:40.142049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/827990525":"2023-07-07T20:15:39.095577+0000","172.21.15.27:6833/827990525":"2023-07-07T20:15:39.095577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:15:47.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f630c06ace0 msgr2=0x7f630c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:47.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f630c06ace0 0x7f630c06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f632c0d0430 tx=0x7f631c008030 comp rx=0 tx=0).stop 2023-07-06T20:15:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 msgr2=0x7f632c0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 0x7f632c0b28c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f631400d800 tx=0x7f631400dbc0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 shutdown_connections 2023-07-06T20:15:47.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f630c06ace0 0x7f630c06d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f632c15fca0 0x7f632c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f632c0b5c90 0x7f632c153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 --2- 172.21.15.27:0/367529999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f632c0b60e0 0x7f632c0b28c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.212+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 >> 172.21.15.27:0/367529999 conn(0x7f632c06c150 msgr2=0x7f632c0b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:47.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.213+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 shutdown_connections 2023-07-06T20:15:47.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:47.213+0000 7f6330fff700 1 -- 172.21.15.27:0/367529999 wait complete. 2023-07-06T20:15:48.234 INFO:tasks.ceph:Scrubbing osd.0 2023-07-06T20:15:48.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:48.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.450+0000 7f5e2b5ff700 1 Processor -- start 2023-07-06T20:15:48.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.450+0000 7f5e2b5ff700 1 -- start start 2023-07-06T20:15:48.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.451+0000 7f5e2b5ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5e240b8440 0x7f5e240cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2b5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e240d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2b5ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e24160cc0 con 0x7f5e240b2180 2023-07-06T20:15:48.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2b5ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e24160ae0 con 0x7f5e240b8440 2023-07-06T20:15:48.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2b5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e24160f80 con 0x7f5e240cfe40 2023-07-06T20:15:48.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5e240b8440 0x7f5e240cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5e240b8440 0x7f5e240cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57720/0 (socket says 172.21.15.27:57720) 2023-07-06T20:15:48.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 -- 172.21.15.27:0/2270497690 learned_addr learned my addr 172.21.15.27:0/2270497690 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:48.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e28b9a700 1 --1- 172.21.15.27:0/2270497690 >> v1:172.21.15.27:6789/0 conn(0x7f5e240b2180 0x7f5e240cf730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50742/0 (socket says 172.21.15.27:50742) 2023-07-06T20:15:48.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e29b9c700 1 --2- 172.21.15.27:0/2270497690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e240d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 -- 172.21.15.27:0/2270497690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 msgr2=0x7f5e240d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:48.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 --2- 172.21.15.27:0/2270497690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e240d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 -- 172.21.15.27:0/2270497690 >> v1:172.21.15.27:6789/0 conn(0x7f5e240b2180 legacy=0x7f5e240cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.452+0000 7f5e2939b700 1 -- 172.21.15.27:0/2270497690 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240cf0e0 con 0x7f5e240b8440 2023-07-06T20:15:48.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.453+0000 7f5e2939b700 1 --2- 172.21.15.27:0/2270497690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b8440 0x7f5e240cd530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5e200098f0 tx=0x7f5e20009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a22a04f860ccb01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.453+0000 7f5e1bfff700 1 -- 172.21.15.27:0/2270497690 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000b8a0 con 0x7f5e240b8440 2023-07-06T20:15:48.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.453+0000 7f5e1bfff700 1 -- 172.21.15.27:0/2270497690 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2000ba00 con 0x7f5e240b8440 2023-07-06T20:15:48.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.453+0000 7f5e1bfff700 1 -- 172.21.15.27:0/2270497690 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000bd90 con 0x7f5e240b8440 2023-07-06T20:15:48.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.453+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2270497690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b8440 msgr2=0x7f5e240cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.453+0000 7f5e2b5ff700 1 --2- 172.21.15.27:0/2270497690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b8440 0x7f5e240cd530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5e200098f0 tx=0x7f5e20009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:48.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.454+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2270497690 shutdown_connections 2023-07-06T20:15:48.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.454+0000 7f5e2b5ff700 1 --2- 172.21.15.27:0/2270497690 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e240d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.454+0000 7f5e2b5ff700 1 --2- 172.21.15.27:0/2270497690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b8440 0x7f5e240cd530 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.454+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2270497690 >> 172.21.15.27:0/2270497690 conn(0x7f5e2406be10 msgr2=0x7f5e24071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:48.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.454+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2270497690 shutdown_connections 2023-07-06T20:15:48.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.454+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2270497690 wait complete. 2023-07-06T20:15:48.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.455+0000 7f5e2b5ff700 1 Processor -- start 2023-07-06T20:15:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.456+0000 7f5e2b5ff700 1 -- start start 2023-07-06T20:15:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.456+0000 7f5e2b5ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b2180 0x7f5e2415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.456+0000 7f5e2b5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e240b8440 0x7f5e241e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2b5ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e241e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2b5ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240cbd60 con 0x7f5e240b2180 2023-07-06T20:15:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2b5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240cb750 con 0x7f5e240b8440 2023-07-06T20:15:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2b5ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2415c2c0 con 0x7f5e240cfe40 2023-07-06T20:15:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b2180 0x7f5e2415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b2180 0x7f5e2415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57726/0 (socket says 172.21.15.27:57726) 2023-07-06T20:15:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 -- 172.21.15.27:0/2805239457 learned_addr learned my addr 172.21.15.27:0/2805239457 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e29b9c700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e241e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e28b9a700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e240b8440 0x7f5e241e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 -- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 msgr2=0x7f5e241e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e241e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 -- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e240b8440 msgr2=0x7f5e241e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e240b8440 0x7f5e241e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 -- 172.21.15.27:0/2805239457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20009550 con 0x7f5e240b2180 2023-07-06T20:15:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e29b9c700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e240cfe40 0x7f5e241e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.457+0000 7f5e2939b700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b2180 0x7f5e2415b7e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.458+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20004480 con 0x7f5e240b2180 2023-07-06T20:15:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.458+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2001f070 con 0x7f5e240b2180 2023-07-06T20:15:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.458+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200038c0 con 0x7f5e240b2180 2023-07-06T20:15:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.458+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e241e79b0 con 0x7f5e240b2180 2023-07-06T20:15:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.458+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e241e73f0 con 0x7f5e240b2180 2023-07-06T20:15:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.459+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e240b8810 con 0x7f5e240b2180 2023-07-06T20:15:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.460+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e20021030 con 0x7f5e240b2180 2023-07-06T20:15:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e19ffb700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e0806ad10 0x7f5e0806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5e2008a740 con 0x7f5e240b2180 2023-07-06T20:15:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e19ffb700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f5e08071180 0x7f5e08073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 --> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e08073c60 con 0x7f5e08071180 2023-07-06T20:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2008cd20 con 0x7f5e240b2180 2023-07-06T20:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e28b9a700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e0806ad10 0x7f5e0806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.461+0000 7f5e29b9c700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f5e08071180 0x7f5e08073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.462+0000 7f5e29b9c700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f5e08071180 0x7f5e08073590 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.462+0000 7f5e28b9a700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e0806ad10 0x7f5e0806d1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e240cb4e0 tx=0x7f5e14007d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.463+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== osd.0 v2:172.21.15.27:6800/323630915 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e08073c60 con 0x7f5e08071180 2023-07-06T20:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 --> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5e240cf3c0 con 0x7f5e08071180 2023-07-06T20:15:48.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e19ffb700 1 -- 172.21.15.27:0/2805239457 <== osd.0 v2:172.21.15.27:6800/323630915 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5e1c009c50 con 0x7f5e08071180 2023-07-06T20:15:48.481 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:48.482 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f5e08071180 msgr2=0x7f5e08073590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:6800/323630915,v1:172.21.15.27:6801/323630915] conn(0x7f5e08071180 0x7f5e08073590 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e0806ad10 msgr2=0x7f5e0806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5e0806ad10 0x7f5e0806d1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e240cb4e0 tx=0x7f5e14007d10 comp rx=0 tx=0).stop 2023-07-06T20:15:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b2180 msgr2=0x7f5e2415b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e2b5ff700 1 --2- 172.21.15.27:0/2805239457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e240b2180 0x7f5e2415b7e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e20004750 comp rx=0 tx=0).stop 2023-07-06T20:15:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.478+0000 7f5e29b9c700 1 -- 172.21.15.27:0/2805239457 reap_dead start 2023-07-06T20:15:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.479+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 shutdown_connections 2023-07-06T20:15:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.479+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 >> 172.21.15.27:0/2805239457 conn(0x7f5e2406be10 msgr2=0x7f5e240b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.479+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 shutdown_connections 2023-07-06T20:15:48.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.479+0000 7f5e2b5ff700 1 -- 172.21.15.27:0/2805239457 wait complete. 2023-07-06T20:15:48.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-06T20:15:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.702+0000 7f032d9ff700 1 Processor -- start 2023-07-06T20:15:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.702+0000 7f032d9ff700 1 -- start start 2023-07-06T20:15:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.703+0000 7f032d9ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f032815fca0 0x7f03280d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.703+0000 7f032d9ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03280b6110 0x7f03280cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.703+0000 7f032d9ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03280cf780 con 0x7f03280b5c90 2023-07-06T20:15:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.703+0000 7f032d9ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280cd490 con 0x7f03280b6110 2023-07-06T20:15:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.703+0000 7f032d9ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280cd770 con 0x7f032815fca0 2023-07-06T20:15:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03280b6110 0x7f03280cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03280b6110 0x7f03280cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57734/0 (socket says 172.21.15.27:57734) 2023-07-06T20:15:48.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 -- 172.21.15.27:0/3795011181 learned_addr learned my addr 172.21.15.27:0/3795011181 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:48.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f0326ffd700 1 --2- 172.21.15.27:0/3795011181 >> v2:172.21.15.153:3300/0 conn(0x7f032815fca0 0x7f03280d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 -- 172.21.15.27:0/3795011181 >> v2:172.21.15.153:3300/0 conn(0x7f032815fca0 msgr2=0x7f03280d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 --2- 172.21.15.27:0/3795011181 >> v2:172.21.15.153:3300/0 conn(0x7f032815fca0 0x7f03280d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 -- 172.21.15.27:0/3795011181 >> v1:172.21.15.153:6789/0 conn(0x7f03280b5c90 legacy=0x7f03280d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 -- 172.21.15.27:0/3795011181 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280cda50 con 0x7f03280b6110 2023-07-06T20:15:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f0326ffd700 1 --2- 172.21.15.27:0/3795011181 >> v2:172.21.15.153:3300/0 conn(0x7f032815fca0 0x7f03280d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.704+0000 7f03277fe700 1 --2- 172.21.15.27:0/3795011181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03280cfc10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f03280d0930 tx=0x7f032000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75820dcc84a96364 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f0325ffb700 1 -- 172.21.15.27:0/3795011181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000e070 con 0x7f03280b6110 2023-07-06T20:15:48.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f0325ffb700 1 -- 172.21.15.27:0/3795011181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032000b9c0 con 0x7f03280b6110 2023-07-06T20:15:48.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f0325ffb700 1 -- 172.21.15.27:0/3795011181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000bd70 con 0x7f03280b6110 2023-07-06T20:15:48.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 -- 172.21.15.27:0/3795011181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 msgr2=0x7f03280cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 --2- 172.21.15.27:0/3795011181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03280cfc10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f03280d0930 tx=0x7f032000b330 comp rx=0 tx=0).stop 2023-07-06T20:15:48.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 -- 172.21.15.27:0/3795011181 shutdown_connections 2023-07-06T20:15:48.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 --2- 172.21.15.27:0/3795011181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03280cfc10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 --2- 172.21.15.27:0/3795011181 >> v2:172.21.15.153:3300/0 conn(0x7f032815fca0 0x7f03280d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 -- 172.21.15.27:0/3795011181 >> 172.21.15.27:0/3795011181 conn(0x7f032806c150 msgr2=0x7f032806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:48.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 -- 172.21.15.27:0/3795011181 shutdown_connections 2023-07-06T20:15:48.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.705+0000 7f032d9ff700 1 -- 172.21.15.27:0/3795011181 wait complete. 2023-07-06T20:15:48.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.707+0000 7f032d9ff700 1 Processor -- start 2023-07-06T20:15:48.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.707+0000 7f032d9ff700 1 -- start start 2023-07-06T20:15:48.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.708+0000 7f032d9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03280b5c90 0x7f0328158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.708+0000 7f032d9ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03281f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.708+0000 7f032d9ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f032815fca0 0x7f03280cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.708+0000 7f032d9ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328159430 con 0x7f03280b6110 2023-07-06T20:15:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.708+0000 7f032d9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03281f2810 con 0x7f03280b5c90 2023-07-06T20:15:48.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.708+0000 7f032d9ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280c7d80 con 0x7f032815fca0 2023-07-06T20:15:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03281f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03281f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57746/0 (socket says 172.21.15.27:57746) 2023-07-06T20:15:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 -- 172.21.15.27:0/1759172931 learned_addr learned my addr 172.21.15.27:0/1759172931 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03277fe700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f032815fca0 0x7f03280cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f0326ffd700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03280b5c90 0x7f0328158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 -- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f032815fca0 msgr2=0x7f03280cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f032815fca0 0x7f03280cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 -- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03280b5c90 msgr2=0x7f0328158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03280b5c90 0x7f0328158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.709+0000 7f03267fc700 1 -- 172.21.15.27:0/1759172931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0320009b30 con 0x7f03280b6110 2023-07-06T20:15:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.710+0000 7f03267fc700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03281f22d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c4e0 tx=0x7f031c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.710+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c0052d0 con 0x7f03280b6110 2023-07-06T20:15:48.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.710+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c005430 con 0x7f03280b6110 2023-07-06T20:15:48.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.710+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c0038c0 con 0x7f03280b6110 2023-07-06T20:15:48.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.710+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03280c8060 con 0x7f03280b6110 2023-07-06T20:15:48.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.710+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03280c8620 con 0x7f03280b6110 2023-07-06T20:15:48.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.711+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03280d1450 con 0x7f03280b6110 2023-07-06T20:15:48.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.712+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f031c003a20 con 0x7f03280b6110 2023-07-06T20:15:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.713+0000 7f030ffff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f031406ace0 0x7f031406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.713+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f031c0884d0 con 0x7f03280b6110 2023-07-06T20:15:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.714+0000 7f0326ffd700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f031406ace0 0x7f031406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.714+0000 7f0326ffd700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f031406ace0 0x7f031406d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f031800c940 tx=0x7f0318008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.717+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f031c053890 con 0x7f03280b6110 2023-07-06T20:15:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.925+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f032806ae40 con 0x7f031406ace0 2023-07-06T20:15:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.926+0000 7f030ffff700 1 -- 172.21.15.27:0/1759172931 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f031800ce60 con 0x7f031406ace0 2023-07-06T20:15:48.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f031406ace0 msgr2=0x7f031406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f031406ace0 0x7f031406d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f031800c940 tx=0x7f0318008040 comp rx=0 tx=0).stop 2023-07-06T20:15:48.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 msgr2=0x7f03281f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:48.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03281f22d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c4e0 tx=0x7f031c00c7f0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 shutdown_connections 2023-07-06T20:15:48.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f031406ace0 0x7f031406d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f032815fca0 0x7f03280cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03280b5c90 0x7f0328158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 --2- 172.21.15.27:0/1759172931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280b6110 0x7f03281f22d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 >> 172.21.15.27:0/1759172931 conn(0x7f032806c150 msgr2=0x7f03280cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.928+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 shutdown_connections 2023-07-06T20:15:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:48.929+0000 7f032d9ff700 1 -- 172.21.15.27:0/1759172931 wait complete. 2023-07-06T20:15:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2023-07-06T20:15:48.949 INFO:tasks.ceph:Scrubbing osd.1 2023-07-06T20:15:48.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:49.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.154+0000 7fe8b97ff700 1 Processor -- start 2023-07-06T20:15:49.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.154+0000 7fe8b97ff700 1 -- start start 2023-07-06T20:15:49.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b97ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b40cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b97ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8b40b2180 0x7fe8b40cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b97ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b4160cc0 con 0x7fe8b40b1c30 2023-07-06T20:15:49.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b97ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b4160ae0 con 0x7fe8b40b2180 2023-07-06T20:15:49.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b97ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4160f80 con 0x7fe8b40b8440 2023-07-06T20:15:49.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8b40b2180 0x7fe8b40cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8b40b2180 0x7fe8b40cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57756/0 (socket says 172.21.15.27:57756) 2023-07-06T20:15:49.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 -- 172.21.15.27:0/2922159549 learned_addr learned my addr 172.21.15.27:0/2922159549 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:49.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 -- 172.21.15.27:0/2922159549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 msgr2=0x7fe8b40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:49.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b2ffd700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b40cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b40cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 -- 172.21.15.27:0/2922159549 >> v1:172.21.15.27:6789/0 conn(0x7fe8b40b1c30 legacy=0x7fe8b40d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8abfff700 1 -- 172.21.15.27:0/2922159549 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40cf0e0 con 0x7fe8b40b2180 2023-07-06T20:15:49.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.157+0000 7fe8b2ffd700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:49.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.156+0000 7fe8b27fc700 1 -- 172.21.15.27:0/2922159549 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236357810 0 0) 0x7fe8b4160cc0 con 0x7fe8b40b1c30 2023-07-06T20:15:49.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.157+0000 7fe8abfff700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b40cf730 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098f0 tx=0x7fe89c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f39448af4a2ab83 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.157+0000 7fe8b27fc700 1 -- 172.21.15.27:0/2922159549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00b8a0 con 0x7fe8b40b2180 2023-07-06T20:15:49.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.157+0000 7fe8b27fc700 1 -- 172.21.15.27:0/2922159549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c00ba00 con 0x7fe8b40b2180 2023-07-06T20:15:49.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.157+0000 7fe8b27fc700 1 -- 172.21.15.27:0/2922159549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00bd90 con 0x7fe8b40b2180 2023-07-06T20:15:49.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.157+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2922159549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 msgr2=0x7fe8b40cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b40cf730 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098f0 tx=0x7fe89c009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:49.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2922159549 shutdown_connections 2023-07-06T20:15:49.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 --2- 172.21.15.27:0/2922159549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b40cf730 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2922159549 >> 172.21.15.27:0/2922159549 conn(0x7fe8b406be10 msgr2=0x7fe8b4071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:49.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2922159549 shutdown_connections 2023-07-06T20:15:49.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.158+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2922159549 wait complete. 2023-07-06T20:15:49.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.159+0000 7fe8b97ff700 1 Processor -- start 2023-07-06T20:15:49.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.160+0000 7fe8b97ff700 1 -- start start 2023-07-06T20:15:49.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.160+0000 7fe8b97ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8b40b1c30 0x7fe8b40cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b97ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b41f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b97ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b97ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b41f6360 con 0x7fe8b40b2180 2023-07-06T20:15:49.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b97ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b41f5da0 con 0x7fe8b40b1c30 2023-07-06T20:15:49.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b97ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b41f6080 con 0x7fe8b40b8440 2023-07-06T20:15:49.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b41f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b41f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57768/0 (socket says 172.21.15.27:57768) 2023-07-06T20:15:49.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 -- 172.21.15.27:0/2086032901 learned_addr learned my addr 172.21.15.27:0/2086032901 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:49.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b37fe700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b2ffd700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8b40b1c30 0x7fe8b40cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 -- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 msgr2=0x7fe8b415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 -- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8b40b1c30 msgr2=0x7fe8b40cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8b40b1c30 0x7fe8b40cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8abfff700 1 -- 172.21.15.27:0/2086032901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c009550 con 0x7fe8b40b2180 2023-07-06T20:15:49.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.161+0000 7fe8b37fe700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8b40b8440 0x7fe8b415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:49.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.162+0000 7fe8abfff700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b41f5370 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009c40 tx=0x7fe89c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.162+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0036a0 con 0x7fe8b40b2180 2023-07-06T20:15:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.162+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c003800 con 0x7fe8b40b2180 2023-07-06T20:15:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.162+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b40c7a20 con 0x7fe8b40b2180 2023-07-06T20:15:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.162+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003b70 con 0x7fe8b40b2180 2023-07-06T20:15:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.162+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b40cc620 con 0x7fe8b40b2180 2023-07-06T20:15:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.163+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8b40b8810 con 0x7fe8b40b2180 2023-07-06T20:15:49.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.164+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe89c003cd0 con 0x7fe8b40b2180 2023-07-06T20:15:49.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.165+0000 7fe8ab7fe700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe89406ace0 0x7fe89406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.165+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe89c021030 con 0x7fe8b40b2180 2023-07-06T20:15:49.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.165+0000 7fe8ab7fe700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7fe894071150 0x7fe894073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.165+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 --> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe894073c30 con 0x7fe894071150 2023-07-06T20:15:49.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.165+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c08ce10 con 0x7fe8b40b2180 2023-07-06T20:15:49.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.166+0000 7fe8b37fe700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7fe894071150 0x7fe894073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.166+0000 7fe8b2ffd700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe89406ace0 0x7fe89406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.166+0000 7fe8b37fe700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7fe894071150 0x7fe894073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.167+0000 7fe8b2ffd700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe89406ace0 0x7fe89406d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4006fd0 tx=0x7fe8a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.167+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== osd.1 v2:172.21.15.27:6824/3450674250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe894073c30 con 0x7fe894071150 2023-07-06T20:15:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.182+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 --> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe8b40cf3c0 con 0x7fe894071150 2023-07-06T20:15:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8ab7fe700 1 -- 172.21.15.27:0/2086032901 <== osd.1 v2:172.21.15.27:6824/3450674250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe8ac001f90 con 0x7fe894071150 2023-07-06T20:15:49.186 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:49.186 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:49.186 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7fe894071150 msgr2=0x7fe894073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:6824/3450674250,v1:172.21.15.27:6825/3450674250] conn(0x7fe894071150 0x7fe894073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe89406ace0 msgr2=0x7fe89406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe89406ace0 0x7fe89406d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4006fd0 tx=0x7fe8a4008040 comp rx=0 tx=0).stop 2023-07-06T20:15:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 msgr2=0x7fe8b41f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 --2- 172.21.15.27:0/2086032901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8b40b2180 0x7fe8b41f5370 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009c40 tx=0x7fe89c004750 comp rx=0 tx=0).stop 2023-07-06T20:15:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b37fe700 1 -- 172.21.15.27:0/2086032901 reap_dead start 2023-07-06T20:15:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 shutdown_connections 2023-07-06T20:15:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 >> 172.21.15.27:0/2086032901 conn(0x7fe8b406be10 msgr2=0x7fe8b40b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 shutdown_connections 2023-07-06T20:15:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.183+0000 7fe8b97ff700 1 -- 172.21.15.27:0/2086032901 wait complete. 2023-07-06T20:15:49.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-06T20:15:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.405+0000 7f41f3bff700 1 Processor -- start 2023-07-06T20:15:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.405+0000 7f41f3bff700 1 -- start start 2023-07-06T20:15:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.406+0000 7f41f3bff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f41ec15fca0 0x7f41ec0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.406+0000 7f41f3bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 0x7f41ec0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.406+0000 7f41f3bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ec0cf780 con 0x7f41ec0b5c90 2023-07-06T20:15:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.406+0000 7f41f3bff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec0cd490 con 0x7f41ec15fca0 2023-07-06T20:15:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.407+0000 7f41f3bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0cd770 con 0x7f41ec0b6110 2023-07-06T20:15:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.407+0000 7f41f219c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 0x7f41ec0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.407+0000 7f41f199b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f41ec15fca0 0x7f41ec0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.407+0000 7f41f219c700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 0x7f41ec0cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.27:55446/0 (socket says 172.21.15.27:55446) 2023-07-06T20:15:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.407+0000 7f41f219c700 1 -- 172.21.15.27:0/1904012239 learned_addr learned my addr 172.21.15.27:0/1904012239 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.407+0000 7f41f119a700 1 --1- 172.21.15.27:0/1904012239 >> v1:172.21.15.153:6789/0 conn(0x7f41ec0b5c90 0x7f41ec0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:44916/0 (socket says 172.21.15.27:44916) 2023-07-06T20:15:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f219c700 1 -- 172.21.15.27:0/1904012239 >> v2:172.21.15.153:3300/0 conn(0x7f41ec15fca0 msgr2=0x7f41ec0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f219c700 1 --2- 172.21.15.27:0/1904012239 >> v2:172.21.15.153:3300/0 conn(0x7f41ec15fca0 0x7f41ec0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f219c700 1 -- 172.21.15.27:0/1904012239 >> v1:172.21.15.153:6789/0 conn(0x7f41ec0b5c90 legacy=0x7f41ec0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f219c700 1 -- 172.21.15.27:0/1904012239 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0cda50 con 0x7f41ec0b6110 2023-07-06T20:15:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f219c700 1 --2- 172.21.15.27:0/1904012239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 0x7f41ec0cfc10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009e50 tx=0x7f41e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220b07a7fc1bd4c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f0999700 1 -- 172.21.15.27:0/1904012239 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400e070 con 0x7f41ec0b6110 2023-07-06T20:15:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f0999700 1 -- 172.21.15.27:0/1904012239 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e400b910 con 0x7f41ec0b6110 2023-07-06T20:15:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.408+0000 7f41f0999700 1 -- 172.21.15.27:0/1904012239 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400bcc0 con 0x7f41ec0b6110 2023-07-06T20:15:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 -- 172.21.15.27:0/1904012239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 msgr2=0x7f41ec0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1904012239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 0x7f41ec0cfc10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009e50 tx=0x7f41e400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 -- 172.21.15.27:0/1904012239 shutdown_connections 2023-07-06T20:15:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1904012239 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec0b6110 0x7f41ec0cfc10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1904012239 >> v2:172.21.15.153:3300/0 conn(0x7f41ec15fca0 0x7f41ec0d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 -- 172.21.15.27:0/1904012239 >> 172.21.15.27:0/1904012239 conn(0x7f41ec06c150 msgr2=0x7f41ec06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 -- 172.21.15.27:0/1904012239 shutdown_connections 2023-07-06T20:15:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.409+0000 7f41f3bff700 1 -- 172.21.15.27:0/1904012239 wait complete. 2023-07-06T20:15:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.411+0000 7f41f3bff700 1 Processor -- start 2023-07-06T20:15:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.412+0000 7f41f3bff700 1 -- start start 2023-07-06T20:15:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f3bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 0x7f41ec159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f3bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41ec0b6110 0x7f41ec15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f3bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec15fca0 0x7f41ec14ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f3bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0c7d60 con 0x7f41ec0b5c90 2023-07-06T20:15:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f3bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec159dd0 con 0x7f41ec0b6110 2023-07-06T20:15:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f3bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0c7470 con 0x7f41ec15fca0 2023-07-06T20:15:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f199b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 0x7f41ec159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f199b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 0x7f41ec159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57776/0 (socket says 172.21.15.27:57776) 2023-07-06T20:15:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f199b700 1 -- 172.21.15.27:0/1721575295 learned_addr learned my addr 172.21.15.27:0/1721575295 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f219c700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec15fca0 0x7f41ec14ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.413+0000 7f41f119a700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41ec0b6110 0x7f41ec15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f199b700 1 -- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec15fca0 msgr2=0x7f41ec14ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f199b700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec15fca0 0x7f41ec14ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f199b700 1 -- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41ec0b6110 msgr2=0x7f41ec15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f199b700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41ec0b6110 0x7f41ec15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f199b700 1 -- 172.21.15.27:0/1721575295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4009b30 con 0x7f41ec0b5c90 2023-07-06T20:15:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f199b700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 0x7f41ec159740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e870 tx=0x7f41e800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8009590 con 0x7f41ec0b5c90 2023-07-06T20:15:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8013680 con 0x7f41ec0b5c90 2023-07-06T20:15:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80139f0 con 0x7f41ec0b5c90 2023-07-06T20:15:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec0c7710 con 0x7f41ec0b5c90 2023-07-06T20:15:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.414+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec0c81f0 con 0x7f41ec0b5c90 2023-07-06T20:15:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.415+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec159f40 con 0x7f41ec0b5c90 2023-07-06T20:15:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.416+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41e8013b50 con 0x7f41ec0b5c90 2023-07-06T20:15:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.417+0000 7f41e27fc700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f41d006ace0 0x7f41d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.417+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f41e8088fe0 con 0x7f41ec0b5c90 2023-07-06T20:15:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.418+0000 7f41f119a700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f41d006ace0 0x7f41d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.419+0000 7f41f119a700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f41d006ace0 0x7f41d006d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0d01c0 tx=0x7f41dc007ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.422+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41e8057e50 con 0x7f41ec0b5c90 2023-07-06T20:15:49.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.626+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f41ec06ae40 con 0x7f41d006ace0 2023-07-06T20:15:49.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.627+0000 7f41e27fc700 1 -- 172.21.15.27:0/1721575295 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f41dc001330 con 0x7f41d006ace0 2023-07-06T20:15:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f41d006ace0 msgr2=0x7f41d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f41d006ace0 0x7f41d006d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0d01c0 tx=0x7f41dc007ed0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 msgr2=0x7f41ec159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 0x7f41ec159740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e870 tx=0x7f41e800eb80 comp rx=0 tx=0).stop 2023-07-06T20:15:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 shutdown_connections 2023-07-06T20:15:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f41d006ace0 0x7f41d006d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41ec15fca0 0x7f41ec14ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41ec0b5c90 0x7f41ec159740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 --2- 172.21.15.27:0/1721575295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41ec0b6110 0x7f41ec15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 >> 172.21.15.27:0/1721575295 conn(0x7f41ec06c150 msgr2=0x7f41ec0d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 shutdown_connections 2023-07-06T20:15:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.630+0000 7f41f3bff700 1 -- 172.21.15.27:0/1721575295 wait complete. 2023-07-06T20:15:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2023-07-06T20:15:49.651 INFO:tasks.ceph:Scrubbing osd.2 2023-07-06T20:15:49.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.857+0000 7f69207ff700 1 Processor -- start 2023-07-06T20:15:49.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.857+0000 7f69207ff700 1 -- start start 2023-07-06T20:15:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.858+0000 7f69207ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69180b8440 0x7f69180cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.858+0000 7f69207ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f69180b2180 0x7f69180cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.858+0000 7f69207ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6918160cc0 con 0x7f69180b1c30 2023-07-06T20:15:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.858+0000 7f69207ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6918160ae0 con 0x7f69180b8440 2023-07-06T20:15:49.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.858+0000 7f69207ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6918160f80 con 0x7f69180b2180 2023-07-06T20:15:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69180b8440 0x7f69180cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69180b8440 0x7f69180cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57782/0 (socket says 172.21.15.27:57782) 2023-07-06T20:15:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 -- 172.21.15.27:0/2327928264 learned_addr learned my addr 172.21.15.27:0/2327928264 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 -- 172.21.15.27:0/2327928264 >> v2:172.21.15.153:3300/0 conn(0x7f69180b2180 msgr2=0x7f69180cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691dd9a700 1 --2- 172.21.15.27:0/2327928264 >> v2:172.21.15.153:3300/0 conn(0x7f69180b2180 0x7f69180cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 --2- 172.21.15.27:0/2327928264 >> v2:172.21.15.153:3300/0 conn(0x7f69180b2180 0x7f69180cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 -- 172.21.15.27:0/2327928264 >> v1:172.21.15.153:6789/0 conn(0x7f69180b1c30 legacy=0x7f69180d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:49.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 -- 172.21.15.27:0/2327928264 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69180cf0e0 con 0x7f69180b8440 2023-07-06T20:15:49.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691dd9a700 1 --2- 172.21.15.27:0/2327928264 >> v2:172.21.15.153:3300/0 conn(0x7f69180b2180 0x7f69180cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:49.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.859+0000 7f691e59b700 1 --2- 172.21.15.27:0/2327928264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b8440 0x7f69180cd530 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f69180b1bc0 tx=0x7f6914009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7f4151e73893c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f691d599700 1 -- 172.21.15.27:0/2327928264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914004230 con 0x7f69180b8440 2023-07-06T20:15:49.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f691d599700 1 -- 172.21.15.27:0/2327928264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6914004390 con 0x7f69180b8440 2023-07-06T20:15:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f691d599700 1 -- 172.21.15.27:0/2327928264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140036a0 con 0x7f69180b8440 2023-07-06T20:15:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 -- 172.21.15.27:0/2327928264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b8440 msgr2=0x7f69180cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 --2- 172.21.15.27:0/2327928264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b8440 0x7f69180cd530 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f69180b1bc0 tx=0x7f6914009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 -- 172.21.15.27:0/2327928264 shutdown_connections 2023-07-06T20:15:49.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 --2- 172.21.15.27:0/2327928264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b8440 0x7f69180cd530 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 --2- 172.21.15.27:0/2327928264 >> v2:172.21.15.153:3300/0 conn(0x7f69180b2180 0x7f69180cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 -- 172.21.15.27:0/2327928264 >> 172.21.15.27:0/2327928264 conn(0x7f691806be10 msgr2=0x7f6918071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 -- 172.21.15.27:0/2327928264 shutdown_connections 2023-07-06T20:15:49.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.860+0000 7f69207ff700 1 -- 172.21.15.27:0/2327928264 wait complete. 2023-07-06T20:15:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.862+0000 7f69207ff700 1 Processor -- start 2023-07-06T20:15:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.862+0000 7f69207ff700 1 -- start start 2023-07-06T20:15:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.863+0000 7f69207ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b1c30 0x7f69181f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.863+0000 7f69207ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69180b2180 0x7f69180cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.863+0000 7f69207ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69180b8440 0x7f6918155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.863+0000 7f69207ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691815dd10 con 0x7f69180b1c30 2023-07-06T20:15:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.863+0000 7f69207ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180caec0 con 0x7f69180b2180 2023-07-06T20:15:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.863+0000 7f69207ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180c9e60 con 0x7f69180b8440 2023-07-06T20:15:49.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b1c30 0x7f69181f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b1c30 0x7f69181f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57792/0 (socket says 172.21.15.27:57792) 2023-07-06T20:15:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 -- 172.21.15.27:0/1406571355 learned_addr learned my addr 172.21.15.27:0/1406571355 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691dd9a700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69180b2180 0x7f69180cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691ed9c700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69180b8440 0x7f6918155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 -- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69180b8440 msgr2=0x7f6918155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f69180b8440 0x7f6918155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 -- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69180b2180 msgr2=0x7f69180cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69180b2180 0x7f69180cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 -- 172.21.15.27:0/1406571355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914009550 con 0x7f69180b1c30 2023-07-06T20:15:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.864+0000 7f691e59b700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b1c30 0x7f69181f3550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f69180d0690 tx=0x7f6914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.865+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914003cd0 con 0x7f69180b1c30 2023-07-06T20:15:49.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.865+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691401f070 con 0x7f69180b1c30 2023-07-06T20:15:49.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.865+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914011870 con 0x7f69180b1c30 2023-07-06T20:15:49.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.865+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69181f3a90 con 0x7f69180b1c30 2023-07-06T20:15:49.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.865+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69180ca0b0 con 0x7f69180b1c30 2023-07-06T20:15:49.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.866+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69180b8810 con 0x7f69180b1c30 2023-07-06T20:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.867+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6914021030 con 0x7f69180b1c30 2023-07-06T20:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.868+0000 7f690effd700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68fc06ace0 0x7f68fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.868+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f691408aa50 con 0x7f69180b1c30 2023-07-06T20:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.868+0000 7f690effd700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7f68fc071150 0x7f68fc073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.868+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 --> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68fc073c30 con 0x7f68fc071150 2023-07-06T20:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.868+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f691408adc0 con 0x7f69180b1c30 2023-07-06T20:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.869+0000 7f691ed9c700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7f68fc071150 0x7f68fc073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.869+0000 7f691dd9a700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68fc06ace0 0x7f68fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.869+0000 7f691ed9c700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7f68fc071150 0x7f68fc073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.869+0000 7f691dd9a700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68fc06ace0 0x7f68fc06d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69180cd340 tx=0x7f6908008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.870+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== osd.2 v2:172.21.15.27:6808/380505300 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f68fc073c30 con 0x7f68fc071150 2023-07-06T20:15:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 --> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f69180cf3c0 con 0x7f68fc071150 2023-07-06T20:15:49.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f690effd700 1 -- 172.21.15.27:0/1406571355 <== osd.2 v2:172.21.15.27:6808/380505300 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6910009a70 con 0x7f68fc071150 2023-07-06T20:15:49.888 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:49.889 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:49.889 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7f68fc071150 msgr2=0x7f68fc073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:6808/380505300,v1:172.21.15.27:6810/380505300] conn(0x7f68fc071150 0x7f68fc073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68fc06ace0 msgr2=0x7f68fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f68fc06ace0 0x7f68fc06d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69180cd340 tx=0x7f6908008040 comp rx=0 tx=0).stop 2023-07-06T20:15:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b1c30 msgr2=0x7f69181f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:49.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 --2- 172.21.15.27:0/1406571355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69180b1c30 0x7f69181f3550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f69180d0690 tx=0x7f6914004750 comp rx=0 tx=0).stop 2023-07-06T20:15:49.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f691ed9c700 1 -- 172.21.15.27:0/1406571355 reap_dead start 2023-07-06T20:15:49.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 shutdown_connections 2023-07-06T20:15:49.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 >> 172.21.15.27:0/1406571355 conn(0x7f691806be10 msgr2=0x7f69180cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:49.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 shutdown_connections 2023-07-06T20:15:49.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:49.886+0000 7f69207ff700 1 -- 172.21.15.27:0/1406571355 wait complete. 2023-07-06T20:15:49.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-06T20:15:50.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.107+0000 7f1b173ff700 1 Processor -- start 2023-07-06T20:15:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.107+0000 7f1b173ff700 1 -- start start 2023-07-06T20:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.108+0000 7f1b173ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b1015fca0 0x7f1b100d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.108+0000 7f1b173ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1b100b5c90 0x7f1b100d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.108+0000 7f1b173ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b100cf780 con 0x7f1b100b6100 2023-07-06T20:15:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.108+0000 7f1b173ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b100cd490 con 0x7f1b100b5c90 2023-07-06T20:15:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.108+0000 7f1b173ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100cd770 con 0x7f1b1015fca0 2023-07-06T20:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.109+0000 7f1b1599c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1b100b6100 0x7f1b100cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:44930/0 (socket says 172.21.15.27:44930) 2023-07-06T20:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.109+0000 7f1b1499a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1b100b5c90 0x7f1b100d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.109+0000 7f1b1599c700 1 -- 172.21.15.27:0/1058204582 learned_addr learned my addr 172.21.15.27:0/1058204582 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:50.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.109+0000 7f1b1519b700 1 --2- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b1015fca0 0x7f1b100d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.109+0000 7f1b07fff700 1 -- 172.21.15.27:0/1058204582 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118257239 0 0) 0x7f1b100cf780 con 0x7f1b100b6100 2023-07-06T20:15:50.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.109+0000 7f1b07fff700 1 -- 172.21.15.27:0/1058204582 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1afc0035b0 con 0x7f1b100b6100 2023-07-06T20:15:50.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b1499a700 1 -- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b1015fca0 msgr2=0x7f1b100d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:50.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b1499a700 1 --2- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b1015fca0 0x7f1b100d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b1499a700 1 -- 172.21.15.27:0/1058204582 >> v1:172.21.15.153:6789/0 conn(0x7f1b100b6100 legacy=0x7f1b100cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b1499a700 1 -- 172.21.15.27:0/1058204582 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100cda50 con 0x7f1b100b5c90 2023-07-06T20:15:50.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b1499a700 1 --2- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b100b5c90 0x7f1b100d2d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009840 tx=0x7f1b00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0012e935b66834 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b07fff700 1 -- 172.21.15.27:0/1058204582 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00011830 con 0x7f1b100b5c90 2023-07-06T20:15:50.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b07fff700 1 -- 172.21.15.27:0/1058204582 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00011990 con 0x7f1b100b5c90 2023-07-06T20:15:50.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.110+0000 7f1b07fff700 1 -- 172.21.15.27:0/1058204582 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00011d20 con 0x7f1b100b5c90 2023-07-06T20:15:50.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 -- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b100b5c90 msgr2=0x7f1b100d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 --2- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b100b5c90 0x7f1b100d2d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009840 tx=0x7f1b00009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:50.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 -- 172.21.15.27:0/1058204582 shutdown_connections 2023-07-06T20:15:50.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 --2- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b1015fca0 0x7f1b100d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 --2- 172.21.15.27:0/1058204582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b100b5c90 0x7f1b100d2d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 -- 172.21.15.27:0/1058204582 >> 172.21.15.27:0/1058204582 conn(0x7f1b1006c150 msgr2=0x7f1b1006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:50.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 -- 172.21.15.27:0/1058204582 shutdown_connections 2023-07-06T20:15:50.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.111+0000 7f1b173ff700 1 -- 172.21.15.27:0/1058204582 wait complete. 2023-07-06T20:15:50.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.113+0000 7f1b173ff700 1 Processor -- start 2023-07-06T20:15:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.113+0000 7f1b173ff700 1 -- start start 2023-07-06T20:15:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b173ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 0x7f1b101f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b173ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b100b6100 0x7f1b100b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b173ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b1015fca0 0x7f1b100b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b173ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100c9f10 con 0x7f1b100b5c90 2023-07-06T20:15:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b173ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100caee0 con 0x7f1b1015fca0 2023-07-06T20:15:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b173ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100caa50 con 0x7f1b100b6100 2023-07-06T20:15:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 0x7f1b101f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 0x7f1b101f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57796/0 (socket says 172.21.15.27:57796) 2023-07-06T20:15:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 -- 172.21.15.27:0/756652653 learned_addr learned my addr 172.21.15.27:0/756652653 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1499a700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b100b6100 0x7f1b100b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1599c700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b1015fca0 0x7f1b100b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 -- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b100b6100 msgr2=0x7f1b100b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b100b6100 0x7f1b100b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 -- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b1015fca0 msgr2=0x7f1b100b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b1015fca0 0x7f1b100b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.114+0000 7f1b1519b700 1 -- 172.21.15.27:0/756652653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000094a0 con 0x7f1b100b5c90 2023-07-06T20:15:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.115+0000 7f1b1519b700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 0x7f1b101f16c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e870 tx=0x7f1b0c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.115+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c009590 con 0x7f1b100b5c90 2023-07-06T20:15:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.115+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c013680 con 0x7f1b100b5c90 2023-07-06T20:15:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.115+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b100ca120 con 0x7f1b100b5c90 2023-07-06T20:15:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.115+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0139f0 con 0x7f1b100b5c90 2023-07-06T20:15:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.115+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b100ca6e0 con 0x7f1b100b5c90 2023-07-06T20:15:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.116+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b100cad30 con 0x7f1b100b5c90 2023-07-06T20:15:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.117+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b0c013b50 con 0x7f1b100b5c90 2023-07-06T20:15:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.118+0000 7f1b05ffb700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1af406ace0 0x7f1af406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.118+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1b0c088ff0 con 0x7f1b100b5c90 2023-07-06T20:15:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.119+0000 7f1b1499a700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1af406ace0 0x7f1af406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.119+0000 7f1b1499a700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1af406ace0 0x7f1af406d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000c00 tx=0x7f1b00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.122+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b0c0bb310 con 0x7f1b100b5c90 2023-07-06T20:15:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.329+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b1006ae40 con 0x7f1af406ace0 2023-07-06T20:15:50.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.330+0000 7f1b05ffb700 1 -- 172.21.15.27:0/756652653 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1b00011630 con 0x7f1af406ace0 2023-07-06T20:15:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1af406ace0 msgr2=0x7f1af406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1af406ace0 0x7f1af406d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000c00 tx=0x7f1b00008030 comp rx=0 tx=0).stop 2023-07-06T20:15:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 msgr2=0x7f1b101f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 0x7f1b101f16c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e870 tx=0x7f1b0c00eb80 comp rx=0 tx=0).stop 2023-07-06T20:15:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 shutdown_connections 2023-07-06T20:15:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f1af406ace0 0x7f1af406d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b1015fca0 0x7f1b100b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b100b5c90 0x7f1b101f16c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 --2- 172.21.15.27:0/756652653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b100b6100 0x7f1b100b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 >> 172.21.15.27:0/756652653 conn(0x7f1b1006c150 msgr2=0x7f1b100b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 shutdown_connections 2023-07-06T20:15:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.333+0000 7f1b173ff700 1 -- 172.21.15.27:0/756652653 wait complete. 2023-07-06T20:15:50.338 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2023-07-06T20:15:50.353 INFO:tasks.ceph:Scrubbing osd.3 2023-07-06T20:15:50.354 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.561+0000 7fe0187ff700 1 Processor -- start 2023-07-06T20:15:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.561+0000 7fe0187ff700 1 -- start start 2023-07-06T20:15:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.562+0000 7fe0187ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe0100b4810 0x7fe0100b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.562+0000 7fe0187ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe0100d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.562+0000 7fe0187ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100b62f0 con 0x7fe01015c6e0 2023-07-06T20:15:50.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe0187ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0100b51d0 con 0x7fe0100b4810 2023-07-06T20:15:50.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe0187ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100d20e0 con 0x7fe0100d1720 2023-07-06T20:15:50.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe0100b4810 0x7fe0100b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe0100b4810 0x7fe0100b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57804/0 (socket says 172.21.15.27:57804) 2023-07-06T20:15:50.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 -- 172.21.15.27:0/1351912703 learned_addr learned my addr 172.21.15.27:0/1351912703 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:50.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe016d9c700 1 --2- 172.21.15.27:0/1351912703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe0100d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 -- 172.21.15.27:0/1351912703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 msgr2=0x7fe0100d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:50.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 --2- 172.21.15.27:0/1351912703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe0100d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 -- 172.21.15.27:0/1351912703 >> v1:172.21.15.27:6789/0 conn(0x7fe01015c6e0 legacy=0x7fe0100b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015d9a700 1 -- 172.21.15.27:0/1351912703 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100d01a0 con 0x7fe0100b4810 2023-07-06T20:15:50.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.563+0000 7fe015599700 1 -- 172.21.15.27:0/1351912703 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507078216 0 0) 0x7fe0100b62f0 con 0x7fe01015c6e0 2023-07-06T20:15:50.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe015d9a700 1 --2- 172.21.15.27:0/1351912703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe0100b4c90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098e0 tx=0x7fe000009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dddf51c3697d69f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe015599700 1 -- 172.21.15.27:0/1351912703 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00000b8a0 con 0x7fe0100b4810 2023-07-06T20:15:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe015599700 1 -- 172.21.15.27:0/1351912703 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00000ba00 con 0x7fe0100b4810 2023-07-06T20:15:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe015599700 1 -- 172.21.15.27:0/1351912703 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00000bdb0 con 0x7fe0100b4810 2023-07-06T20:15:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe0187ff700 1 -- 172.21.15.27:0/1351912703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 msgr2=0x7fe0100b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe0187ff700 1 --2- 172.21.15.27:0/1351912703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe0100b4c90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098e0 tx=0x7fe000009bf0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe0187ff700 1 -- 172.21.15.27:0/1351912703 shutdown_connections 2023-07-06T20:15:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe0187ff700 1 --2- 172.21.15.27:0/1351912703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe0100d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe0187ff700 1 --2- 172.21.15.27:0/1351912703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe0100b4c90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.564+0000 7fe0187ff700 1 -- 172.21.15.27:0/1351912703 >> 172.21.15.27:0/1351912703 conn(0x7fe010072320 msgr2=0x7fe01006a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.565+0000 7fe0187ff700 1 -- 172.21.15.27:0/1351912703 shutdown_connections 2023-07-06T20:15:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.565+0000 7fe0187ff700 1 -- 172.21.15.27:0/1351912703 wait complete. 2023-07-06T20:15:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.566+0000 7fe0187ff700 1 Processor -- start 2023-07-06T20:15:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.566+0000 7fe0187ff700 1 -- start start 2023-07-06T20:15:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.567+0000 7fe0187ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe01014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.567+0000 7fe0187ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe01014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.567+0000 7fe0187ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01015c6e0 0x7fe01015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe0187ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01014a870 con 0x7fe0100b4810 2023-07-06T20:15:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe0187ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c9f00 con 0x7fe01015c6e0 2023-07-06T20:15:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe0187ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100ca1e0 con 0x7fe0100d1720 2023-07-06T20:15:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe01014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe01014a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57808/0 (socket says 172.21.15.27:57808) 2023-07-06T20:15:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 -- 172.21.15.27:0/4286895616 learned_addr learned my addr 172.21.15.27:0/4286895616 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe015d9a700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe01014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe016d9c700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01015c6e0 0x7fe01015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 -- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 msgr2=0x7fe01014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe01014ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 -- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01015c6e0 msgr2=0x7fe01015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01015c6e0 0x7fe01015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 -- 172.21.15.27:0/4286895616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000094f0 con 0x7fe0100b4810 2023-07-06T20:15:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe015d9a700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100d1720 0x7fe01014ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.568+0000 7fe01659b700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe01014a330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c550 tx=0x7fe00c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.569+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0054e0 con 0x7fe0100b4810 2023-07-06T20:15:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.569+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c003680 con 0x7fe0100b4810 2023-07-06T20:15:50.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.569+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0100ca4c0 con 0x7fe0100b4810 2023-07-06T20:15:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.569+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c003a30 con 0x7fe0100b4810 2023-07-06T20:15:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.569+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0100caa80 con 0x7fe0100b4810 2023-07-06T20:15:50.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.570+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0100d0c40 con 0x7fe0100b4810 2023-07-06T20:15:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.571+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe00c003b90 con 0x7fe0100b4810 2023-07-06T20:15:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.572+0000 7fe006ffd700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdff406ace0 0x7fdff406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.572+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe00c088b20 con 0x7fe0100b4810 2023-07-06T20:15:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.572+0000 7fe006ffd700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7fdff4071150 0x7fdff4073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.572+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 --> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdff4073c30 con 0x7fdff4071150 2023-07-06T20:15:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.572+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe00c088e90 con 0x7fe0100b4810 2023-07-06T20:15:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.572+0000 7fe016d9c700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7fdff4071150 0x7fdff4073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.573+0000 7fe015d9a700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdff406ace0 0x7fdff406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.573+0000 7fe016d9c700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7fdff4071150 0x7fdff4073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.573+0000 7fe015d9a700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdff406ace0 0x7fdff406d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe000009130 tx=0x7fe000008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.574+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== osd.3 v2:172.21.15.27:6809/2399591703 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdff4073c30 con 0x7fdff4071150 2023-07-06T20:15:50.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.589+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 --> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe0100d0960 con 0x7fdff4071150 2023-07-06T20:15:50.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.589+0000 7fe006ffd700 1 -- 172.21.15.27:0/4286895616 <== osd.3 v2:172.21.15.27:6809/2399591703 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe008009b70 con 0x7fdff4071150 2023-07-06T20:15:50.591 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:50.591 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:50.592 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:50.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.589+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7fdff4071150 msgr2=0x7fdff4073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.589+0000 7fe0187ff700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:6809/2399591703,v1:172.21.15.27:6811/2399591703] conn(0x7fdff4071150 0x7fdff4073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.589+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdff406ace0 msgr2=0x7fdff406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fdff406ace0 0x7fdff406d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe000009130 tx=0x7fe000008030 comp rx=0 tx=0).stop 2023-07-06T20:15:50.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 msgr2=0x7fe01014a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 --2- 172.21.15.27:0/4286895616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0100b4810 0x7fe01014a330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c550 tx=0x7fe00c00c910 comp rx=0 tx=0).stop 2023-07-06T20:15:50.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe016d9c700 1 -- 172.21.15.27:0/4286895616 reap_dead start 2023-07-06T20:15:50.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 shutdown_connections 2023-07-06T20:15:50.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 >> 172.21.15.27:0/4286895616 conn(0x7fe010072320 msgr2=0x7fe0100b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:50.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 shutdown_connections 2023-07-06T20:15:50.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.590+0000 7fe0187ff700 1 -- 172.21.15.27:0/4286895616 wait complete. 2023-07-06T20:15:50.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-06T20:15:50.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:15:50.799 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:15:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.813+0000 7f33bd1ff700 1 Processor -- start 2023-07-06T20:15:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.813+0000 7f33bd1ff700 1 -- start start 2023-07-06T20:15:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33bd1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f33b815fca0 0x7f33b80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33bd1ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f33b80b5c90 0x7f33b80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33bd1ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b80cf780 con 0x7f33b80b6110 2023-07-06T20:15:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33bd1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b80cd490 con 0x7f33b815fca0 2023-07-06T20:15:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33bd1ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b80cd770 con 0x7f33b80b5c90 2023-07-06T20:15:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b6d9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f33b815fca0 0x7f33b80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b759e700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f33b80b6110 0x7f33b80cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50780/0 (socket says 172.21.15.27:50780) 2023-07-06T20:15:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b6d9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f33b815fca0 0x7f33b80d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57814/0 (socket says 172.21.15.27:57814) 2023-07-06T20:15:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b759e700 1 -- 172.21.15.27:0/3770951990 learned_addr learned my addr 172.21.15.27:0/3770951990 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b659c700 1 --2- 172.21.15.27:0/3770951990 >> v2:172.21.15.153:3300/0 conn(0x7f33b80b5c90 0x7f33b80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b6d9d700 1 -- 172.21.15.27:0/3770951990 >> v2:172.21.15.153:3300/0 conn(0x7f33b80b5c90 msgr2=0x7f33b80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b6d9d700 1 --2- 172.21.15.27:0/3770951990 >> v2:172.21.15.153:3300/0 conn(0x7f33b80b5c90 0x7f33b80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b6d9d700 1 -- 172.21.15.27:0/3770951990 >> v1:172.21.15.27:6789/0 conn(0x7f33b80b6110 legacy=0x7f33b80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.816+0000 7f33b6d9d700 1 -- 172.21.15.27:0/3770951990 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80cda50 con 0x7f33b815fca0 2023-07-06T20:15:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.815+0000 7f33b5d9b700 1 -- 172.21.15.27:0/3770951990 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655193133 0 0) 0x7f33b80cf780 con 0x7f33b80b6110 2023-07-06T20:15:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.816+0000 7f33b6d9d700 1 --2- 172.21.15.27:0/3770951990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b815fca0 0x7f33b80d1420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f33b806b6f0 tx=0x7f33a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f718aaa08c75437d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.816+0000 7f33b5d9b700 1 -- 172.21.15.27:0/3770951990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8011830 con 0x7f33b815fca0 2023-07-06T20:15:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.816+0000 7f33b5d9b700 1 -- 172.21.15.27:0/3770951990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a8011990 con 0x7f33b815fca0 2023-07-06T20:15:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.816+0000 7f33b5d9b700 1 -- 172.21.15.27:0/3770951990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8011d20 con 0x7f33b815fca0 2023-07-06T20:15:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3770951990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b815fca0 msgr2=0x7f33b80d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3770951990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b815fca0 0x7f33b80d1420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f33b806b6f0 tx=0x7f33a8009b20 comp rx=0 tx=0).stop 2023-07-06T20:15:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3770951990 shutdown_connections 2023-07-06T20:15:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3770951990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b815fca0 0x7f33b80d1420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3770951990 >> v2:172.21.15.153:3300/0 conn(0x7f33b80b5c90 0x7f33b80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3770951990 >> 172.21.15.27:0/3770951990 conn(0x7f33b806c150 msgr2=0x7f33b806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3770951990 shutdown_connections 2023-07-06T20:15:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.817+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3770951990 wait complete. 2023-07-06T20:15:50.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.819+0000 7f33bd1ff700 1 Processor -- start 2023-07-06T20:15:50.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.819+0000 7f33bd1ff700 1 -- start start 2023-07-06T20:15:50.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33bd1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33b80b5c90 0x7f33b81f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33bd1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 0x7f33b81f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33bd1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b815fca0 0x7f33b80cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33bd1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81f6070 con 0x7f33b80b6110 2023-07-06T20:15:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33bd1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81f1a00 con 0x7f33b815fca0 2023-07-06T20:15:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33bd1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80ca280 con 0x7f33b80b5c90 2023-07-06T20:15:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 0x7f33b81f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 0x7f33b81f14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57824/0 (socket says 172.21.15.27:57824) 2023-07-06T20:15:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33b659c700 1 -- 172.21.15.27:0/3929677103 learned_addr learned my addr 172.21.15.27:0/3929677103 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33b6d9d700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33b80b5c90 0x7f33b81f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.820+0000 7f33b759e700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b815fca0 0x7f33b80cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33b659c700 1 -- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33b80b5c90 msgr2=0x7f33b81f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33b659c700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33b80b5c90 0x7f33b81f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33b659c700 1 -- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b815fca0 msgr2=0x7f33b80cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33b659c700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b815fca0 0x7f33b80cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33b659c700 1 -- 172.21.15.27:0/3929677103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80094a0 con 0x7f33b80b6110 2023-07-06T20:15:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33b659c700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 0x7f33b81f14c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00e9b0 tx=0x7f33ac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac009710 con 0x7f33b80b6110 2023-07-06T20:15:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac00c850 con 0x7f33b80b6110 2023-07-06T20:15:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00cbc0 con 0x7f33b80b6110 2023-07-06T20:15:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.821+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b80ca560 con 0x7f33b80b6110 2023-07-06T20:15:50.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.822+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b80cab20 con 0x7f33b80b6110 2023-07-06T20:15:50.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.823+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b80008d0 con 0x7f33b80b6110 2023-07-06T20:15:50.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.824+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33ac00cd20 con 0x7f33b80b6110 2023-07-06T20:15:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.825+0000 7f339f7fe700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f33a406ace0 0x7f33a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.825+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f33ac089210 con 0x7f33b80b6110 2023-07-06T20:15:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.825+0000 7f33b6d9d700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f33a406ace0 0x7f33a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.826+0000 7f33b6d9d700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f33a406ace0 0x7f33a406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f33a80090b0 tx=0x7f33a801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:50.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:50.829+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f33ac0545c0 con 0x7f33b80b6110 2023-07-06T20:15:51.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.041+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f33b806ae40 con 0x7f33a406ace0 2023-07-06T20:15:51.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.042+0000 7f339f7fe700 1 -- 172.21.15.27:0/3929677103 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f33a8011b80 con 0x7f33a406ace0 2023-07-06T20:15:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f33a406ace0 msgr2=0x7f33a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f33a406ace0 0x7f33a406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f33a80090b0 tx=0x7f33a801c040 comp rx=0 tx=0).stop 2023-07-06T20:15:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 msgr2=0x7f33b81f14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 0x7f33b81f14c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00e9b0 tx=0x7f33ac00ecc0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 shutdown_connections 2023-07-06T20:15:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f33a406ace0 0x7f33a406d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33b815fca0 0x7f33b80cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33b80b5c90 0x7f33b81f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 --2- 172.21.15.27:0/3929677103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b80b6110 0x7f33b81f14c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 >> 172.21.15.27:0/3929677103 conn(0x7f33b806c150 msgr2=0x7f33b80d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 shutdown_connections 2023-07-06T20:15:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.044+0000 7f33bd1ff700 1 -- 172.21.15.27:0/3929677103 wait complete. 2023-07-06T20:15:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2023-07-06T20:15:51.064 INFO:tasks.ceph:Scrubbing osd.4 2023-07-06T20:15:51.065 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:51.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.272+0000 7feffcdff700 1 Processor -- start 2023-07-06T20:15:51.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.272+0000 7feffcdff700 1 -- start start 2023-07-06T20:15:51.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feffcdff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feff80b8440 0x7feff80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feffcdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b2180 0x7feff80cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feffcdff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff8160cc0 con 0x7feff80b1c30 2023-07-06T20:15:51.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feffcdff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff8160ae0 con 0x7feff80b8440 2023-07-06T20:15:51.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feffcdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff8160f80 con 0x7feff80b2180 2023-07-06T20:15:51.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feff659c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feff80b8440 0x7feff80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.273+0000 7feff659c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feff80b8440 0x7feff80cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57828/0 (socket says 172.21.15.27:57828) 2023-07-06T20:15:51.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff659c700 1 -- 172.21.15.27:0/3998649796 learned_addr learned my addr 172.21.15.27:0/3998649796 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:51.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff5d9b700 1 --2- 172.21.15.27:0/3998649796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b2180 0x7feff80cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff659c700 1 -- 172.21.15.27:0/3998649796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b2180 msgr2=0x7feff80cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:51.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff659c700 1 --2- 172.21.15.27:0/3998649796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b2180 0x7feff80cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff659c700 1 -- 172.21.15.27:0/3998649796 >> v1:172.21.15.153:6789/0 conn(0x7feff80b1c30 legacy=0x7feff80d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff659c700 1 -- 172.21.15.27:0/3998649796 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80cf0e0 con 0x7feff80b8440 2023-07-06T20:15:51.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.274+0000 7feff659c700 1 --2- 172.21.15.27:0/3998649796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b8440 0x7feff80cd530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fefe80098f0 tx=0x7fefe8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c53a5b57e01958 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feff559a700 1 -- 172.21.15.27:0/3998649796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011830 con 0x7feff80b8440 2023-07-06T20:15:51.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feff559a700 1 -- 172.21.15.27:0/3998649796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8011990 con 0x7feff80b8440 2023-07-06T20:15:51.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feff559a700 1 -- 172.21.15.27:0/3998649796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011d20 con 0x7feff80b8440 2023-07-06T20:15:51.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 -- 172.21.15.27:0/3998649796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b8440 msgr2=0x7feff80cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 --2- 172.21.15.27:0/3998649796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b8440 0x7feff80cd530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fefe80098f0 tx=0x7fefe8009c00 comp rx=0 tx=0).stop 2023-07-06T20:15:51.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 -- 172.21.15.27:0/3998649796 shutdown_connections 2023-07-06T20:15:51.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 --2- 172.21.15.27:0/3998649796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b8440 0x7feff80cd530 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 --2- 172.21.15.27:0/3998649796 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b2180 0x7feff80cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 -- 172.21.15.27:0/3998649796 >> 172.21.15.27:0/3998649796 conn(0x7feff806be10 msgr2=0x7feff8071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 -- 172.21.15.27:0/3998649796 shutdown_connections 2023-07-06T20:15:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.275+0000 7feffcdff700 1 -- 172.21.15.27:0/3998649796 wait complete. 2023-07-06T20:15:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.277+0000 7feffcdff700 1 Processor -- start 2023-07-06T20:15:51.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.277+0000 7feffcdff700 1 -- start start 2023-07-06T20:15:51.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feffcdff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b1c30 0x7feff8155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feffcdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feff80b2180 0x7feff815f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feffcdff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b8440 0x7feff81f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feffcdff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80c9800 con 0x7feff80b1c30 2023-07-06T20:15:51.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feffcdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff81e7d60 con 0x7feff80b2180 2023-07-06T20:15:51.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feffcdff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff81e7150 con 0x7feff80b8440 2023-07-06T20:15:51.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feff659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b1c30 0x7feff8155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feff659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b1c30 0x7feff8155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57844/0 (socket says 172.21.15.27:57844) 2023-07-06T20:15:51.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.278+0000 7feff659c700 1 -- 172.21.15.27:0/348851493 learned_addr learned my addr 172.21.15.27:0/348851493 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:51.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff659c700 1 -- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b8440 msgr2=0x7feff81f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff6d9d700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b8440 0x7feff81f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff659c700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b8440 0x7feff81f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff5d9b700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feff80b2180 0x7feff815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff659c700 1 -- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feff80b2180 msgr2=0x7feff815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff659c700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feff80b2180 0x7feff815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff659c700 1 -- 172.21.15.27:0/348851493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe8009550 con 0x7feff80b1c30 2023-07-06T20:15:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff6d9d700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feff80b8440 0x7feff81f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff5d9b700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feff80b2180 0x7feff815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feff659c700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b1c30 0x7feff8155e10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009c40 tx=0x7fefe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80036a0 con 0x7feff80b1c30 2023-07-06T20:15:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe801e070 con 0x7feff80b1c30 2023-07-06T20:15:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff81e7430 con 0x7feff80b1c30 2023-07-06T20:15:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.279+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8003a60 con 0x7feff80b1c30 2023-07-06T20:15:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.280+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff81e79f0 con 0x7feff80b1c30 2023-07-06T20:15:51.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.280+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feff80b8810 con 0x7feff80b1c30 2023-07-06T20:15:51.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.282+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefe8003bc0 con 0x7feff80b1c30 2023-07-06T20:15:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7fefe6ffd700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fefd806ace0 0x7fefd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fefe808a710 con 0x7feff80b1c30 2023-07-06T20:15:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7fefe6ffd700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fefd8071150 0x7fefd8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 --> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefd8073c30 con 0x7fefd8071150 2023-07-06T20:15:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefe808aa80 con 0x7feff80b1c30 2023-07-06T20:15:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7feff5d9b700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fefd806ace0 0x7fefd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.283+0000 7feff6d9d700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fefd8071150 0x7fefd8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.284+0000 7feff5d9b700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fefd806ace0 0x7fefd806d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feff80d0a80 tx=0x7fefec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.284+0000 7feff6d9d700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fefd8071150 0x7fefd8073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.285+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== osd.4 v2:172.21.15.153:6800/2660030028 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fefd8073c30 con 0x7fefd8071150 2023-07-06T20:15:51.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.300+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 --> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feff80cf3c0 con 0x7fefd8071150 2023-07-06T20:15:51.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.300+0000 7fefe6ffd700 1 -- 172.21.15.27:0/348851493 <== osd.4 v2:172.21.15.153:6800/2660030028 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feff0012380 con 0x7fefd8071150 2023-07-06T20:15:51.302 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:51.303 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:51.303 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:51.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fefd8071150 msgr2=0x7fefd8073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6800/2660030028,v1:172.21.15.153:6801/2660030028] conn(0x7fefd8071150 0x7fefd8073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fefd806ace0 msgr2=0x7fefd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fefd806ace0 0x7fefd806d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feff80d0a80 tx=0x7fefec008040 comp rx=0 tx=0).stop 2023-07-06T20:15:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b1c30 msgr2=0x7feff8155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 --2- 172.21.15.27:0/348851493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff80b1c30 0x7feff8155e10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009c40 tx=0x7fefe8004750 comp rx=0 tx=0).stop 2023-07-06T20:15:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feff6d9d700 1 -- 172.21.15.27:0/348851493 reap_dead start 2023-07-06T20:15:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 shutdown_connections 2023-07-06T20:15:51.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 >> 172.21.15.27:0/348851493 conn(0x7feff806be10 msgr2=0x7feff80d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:51.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 shutdown_connections 2023-07-06T20:15:51.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.301+0000 7feffcdff700 1 -- 172.21.15.27:0/348851493 wait complete. 2023-07-06T20:15:51.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-06T20:15:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.523+0000 7f5723fff700 1 Processor -- start 2023-07-06T20:15:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.523+0000 7f5723fff700 1 -- start start 2023-07-06T20:15:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.524+0000 7f5723fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c0b6110 0x7f571c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.524+0000 7f5723fff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0cf780 con 0x7f571c0b5c90 2023-07-06T20:15:51.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.524+0000 7f5723fff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0cd490 con 0x7f571c15fca0 2023-07-06T20:15:51.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.524+0000 7f5723fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0cd770 con 0x7f571c0b6110 2023-07-06T20:15:51.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f572159a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f571c0b5c90 0x7f571c0d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50796/0 (socket says 172.21.15.27:50796) 2023-07-06T20:15:51.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f572159a700 1 -- 172.21.15.27:0/3723090444 learned_addr learned my addr 172.21.15.27:0/3723090444 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f5721d9b700 1 --1- 172.21.15.27:0/3723090444 >> v1:172.21.15.153:6789/0 conn(0x7f571c15fca0 0x7f571c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:44950/0 (socket says 172.21.15.27:44950) 2023-07-06T20:15:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f572259c700 1 --2- 172.21.15.27:0/3723090444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c0b6110 0x7f571c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f5720d99700 1 -- 172.21.15.27:0/3723090444 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745931355 0 0) 0x7f571c0cf780 con 0x7f571c0b5c90 2023-07-06T20:15:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f5720d99700 1 -- 172.21.15.27:0/3723090444 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57080035b0 con 0x7f571c0b5c90 2023-07-06T20:15:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f5720d99700 1 -- 172.21.15.27:0/3723090444 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022327574 0 0) 0x7f571c0cd490 con 0x7f571c15fca0 2023-07-06T20:15:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f572259c700 1 -- 172.21.15.27:0/3723090444 >> v1:172.21.15.153:6789/0 conn(0x7f571c15fca0 legacy=0x7f571c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f572259c700 1 -- 172.21.15.27:0/3723090444 >> v1:172.21.15.27:6789/0 conn(0x7f571c0b5c90 legacy=0x7f571c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.525+0000 7f572259c700 1 -- 172.21.15.27:0/3723090444 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0cda50 con 0x7f571c0b6110 2023-07-06T20:15:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f572259c700 1 --2- 172.21.15.27:0/3723090444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c0b6110 0x7f571c0cfc10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5714009dc0 tx=0x7f571400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69be1b7ed90762c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5720d99700 1 -- 172.21.15.27:0/3723090444 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400e070 con 0x7f571c0b6110 2023-07-06T20:15:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5720d99700 1 -- 172.21.15.27:0/3723090444 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571400bb20 con 0x7f571c0b6110 2023-07-06T20:15:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5720d99700 1 -- 172.21.15.27:0/3723090444 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400bed0 con 0x7f571c0b6110 2023-07-06T20:15:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5723fff700 1 -- 172.21.15.27:0/3723090444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c0b6110 msgr2=0x7f571c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5723fff700 1 --2- 172.21.15.27:0/3723090444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c0b6110 0x7f571c0cfc10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5714009dc0 tx=0x7f571400b330 comp rx=0 tx=0).stop 2023-07-06T20:15:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5723fff700 1 -- 172.21.15.27:0/3723090444 shutdown_connections 2023-07-06T20:15:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5723fff700 1 --2- 172.21.15.27:0/3723090444 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c0b6110 0x7f571c0cfc10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.526+0000 7f5723fff700 1 -- 172.21.15.27:0/3723090444 >> 172.21.15.27:0/3723090444 conn(0x7f571c06c150 msgr2=0x7f571c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:51.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.527+0000 7f5723fff700 1 -- 172.21.15.27:0/3723090444 shutdown_connections 2023-07-06T20:15:51.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.527+0000 7f5723fff700 1 -- 172.21.15.27:0/3723090444 wait complete. 2023-07-06T20:15:51.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.528+0000 7f5723fff700 1 Processor -- start 2023-07-06T20:15:51.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.528+0000 7f5723fff700 1 -- start start 2023-07-06T20:15:51.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.529+0000 7f5723fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f571c0b5c90 0x7f571c15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.529+0000 7f5723fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 0x7f571c1efb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.529+0000 7f5723fff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c15fca0 0x7f571c1f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.529+0000 7f5723fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c1546e0 con 0x7f571c0b6110 2023-07-06T20:15:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.529+0000 7f5723fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ca020 con 0x7f571c0b5c90 2023-07-06T20:15:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.529+0000 7f5723fff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ca300 con 0x7f571c15fca0 2023-07-06T20:15:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 0x7f571c1efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 0x7f571c1efb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57858/0 (socket says 172.21.15.27:57858) 2023-07-06T20:15:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 -- 172.21.15.27:0/2551663210 learned_addr learned my addr 172.21.15.27:0/2551663210 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572259c700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c15fca0 0x7f571c1f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f5721d9b700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f571c0b5c90 0x7f571c15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 -- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c15fca0 msgr2=0x7f571c1f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c15fca0 0x7f571c1f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 -- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f571c0b5c90 msgr2=0x7f571c15eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f571c0b5c90 0x7f571c15eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.530+0000 7f572159a700 1 -- 172.21.15.27:0/2551663210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5714009a70 con 0x7f571c0b6110 2023-07-06T20:15:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.531+0000 7f572159a700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 0x7f571c1efb90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c4d0 tx=0x7f570c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.531+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c005d70 con 0x7f571c0b6110 2023-07-06T20:15:51.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.531+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c0070d0 con 0x7f571c0b6110 2023-07-06T20:15:51.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.531+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c0ca5e0 con 0x7f571c0b6110 2023-07-06T20:15:51.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.531+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c007440 con 0x7f571c0b6110 2023-07-06T20:15:51.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.531+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c15f090 con 0x7f571c0b6110 2023-07-06T20:15:51.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.532+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c0cab00 con 0x7f571c0b6110 2023-07-06T20:15:51.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.533+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f570c0075a0 con 0x7f571c0b6110 2023-07-06T20:15:51.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.534+0000 7f57127fc700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f570006ace0 0x7f570006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.534+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f570c088440 con 0x7f571c0b6110 2023-07-06T20:15:51.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.535+0000 7f5721d9b700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f570006ace0 0x7f570006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.535+0000 7f5721d9b700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f570006ace0 0x7f570006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5718000f60 tx=0x7f5718007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.539+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f571c0cab00 con 0x7f571c0b6110 2023-07-06T20:15:51.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.744+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f571c06ae40 con 0x7f570006ace0 2023-07-06T20:15:51.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.745+0000 7f57127fc700 1 -- 172.21.15.27:0/2551663210 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f57180133f0 con 0x7f570006ace0 2023-07-06T20:15:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.747+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f570006ace0 msgr2=0x7f570006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.747+0000 7f5723fff700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f570006ace0 0x7f570006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5718000f60 tx=0x7f5718007e70 comp rx=0 tx=0).stop 2023-07-06T20:15:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.747+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 msgr2=0x7f571c1efb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.747+0000 7f5723fff700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 0x7f571c1efb90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c4d0 tx=0x7f570c00c7e0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 shutdown_connections 2023-07-06T20:15:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f570006ace0 0x7f570006d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f571c15fca0 0x7f571c1f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f571c0b5c90 0x7f571c15eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 --2- 172.21.15.27:0/2551663210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f571c0b6110 0x7f571c1efb90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 >> 172.21.15.27:0/2551663210 conn(0x7f571c06c150 msgr2=0x7f571c0d2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:51.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 shutdown_connections 2023-07-06T20:15:51.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.748+0000 7f5723fff700 1 -- 172.21.15.27:0/2551663210 wait complete. 2023-07-06T20:15:51.753 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2023-07-06T20:15:51.767 INFO:tasks.ceph:Scrubbing osd.5 2023-07-06T20:15:51.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.974+0000 7f14e59ff700 1 Processor -- start 2023-07-06T20:15:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.975+0000 7f14e59ff700 1 -- start start 2023-07-06T20:15:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14e59ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14e00cfe40 0x7f14e00cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14e59ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e0160cc0 con 0x7f14e00b8440 2023-07-06T20:15:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14e59ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e0160f80 con 0x7f14e00b2180 2023-07-06T20:15:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14e59ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e0160ae0 con 0x7f14e00cfe40 2023-07-06T20:15:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14deffd700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f14e00b8440 0x7f14e00cf730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50252/0 (socket says 172.21.15.27:50252) 2023-07-06T20:15:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14deffd700 1 -- 172.21.15.27:0/2437833702 learned_addr learned my addr 172.21.15.27:0/2437833702 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.976+0000 7f14d7fff700 1 --2- 172.21.15.27:0/2437833702 >> v2:172.21.15.27:3300/0 conn(0x7f14e00cfe40 0x7f14e00cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14de7fc700 1 -- 172.21.15.27:0/2437833702 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716719819 0 0) 0x7f14e0160cc0 con 0x7f14e00b8440 2023-07-06T20:15:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14de7fc700 1 -- 172.21.15.27:0/2437833702 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14cc0035b0 con 0x7f14e00b8440 2023-07-06T20:15:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14d7fff700 1 -- 172.21.15.27:0/2437833702 >> v1:172.21.15.153:6789/0 conn(0x7f14e00b2180 legacy=0x7f14e00d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14d7fff700 1 -- 172.21.15.27:0/2437833702 >> v1:172.21.15.27:6789/0 conn(0x7f14e00b8440 legacy=0x7f14e00cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14d7fff700 1 -- 172.21.15.27:0/2437833702 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00cf0e0 con 0x7f14e00cfe40 2023-07-06T20:15:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14d7fff700 1 --2- 172.21.15.27:0/2437833702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00cfe40 0x7f14e00cd530 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009840 tx=0x7f14c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e61b1816626828 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14de7fc700 1 -- 172.21.15.27:0/2437833702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8011830 con 0x7f14e00cfe40 2023-07-06T20:15:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14de7fc700 1 -- 172.21.15.27:0/2437833702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8011990 con 0x7f14e00cfe40 2023-07-06T20:15:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.977+0000 7f14de7fc700 1 -- 172.21.15.27:0/2437833702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8011d20 con 0x7f14e00cfe40 2023-07-06T20:15:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 -- 172.21.15.27:0/2437833702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00cfe40 msgr2=0x7f14e00cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 --2- 172.21.15.27:0/2437833702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00cfe40 0x7f14e00cd530 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009840 tx=0x7f14c8009b50 comp rx=0 tx=0).stop 2023-07-06T20:15:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 -- 172.21.15.27:0/2437833702 shutdown_connections 2023-07-06T20:15:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 --2- 172.21.15.27:0/2437833702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00cfe40 0x7f14e00cd530 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 -- 172.21.15.27:0/2437833702 >> 172.21.15.27:0/2437833702 conn(0x7f14e006be10 msgr2=0x7f14e0071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 -- 172.21.15.27:0/2437833702 shutdown_connections 2023-07-06T20:15:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.978+0000 7f14e59ff700 1 -- 172.21.15.27:0/2437833702 wait complete. 2023-07-06T20:15:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.980+0000 7f14e59ff700 1 Processor -- start 2023-07-06T20:15:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.980+0000 7f14e59ff700 1 -- start start 2023-07-06T20:15:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14e59ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00b2180 0x7f14e0149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14e59ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e00b8440 0x7f14e00c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14e59ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e00cfe40 0x7f14e00c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14e59ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e015d2c0 con 0x7f14e00b2180 2023-07-06T20:15:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14e59ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00cc860 con 0x7f14e00b8440 2023-07-06T20:15:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14e59ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0149760 con 0x7f14e00cfe40 2023-07-06T20:15:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00b2180 0x7f14e0149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00b2180 0x7f14e0149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45844/0 (socket says 172.21.15.27:45844) 2023-07-06T20:15:51.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 -- 172.21.15.27:0/485503249 learned_addr learned my addr 172.21.15.27:0/485503249 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:51.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14df7fe700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e00cfe40 0x7f14e00c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14d7fff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e00b8440 0x7f14e00c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 -- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e00cfe40 msgr2=0x7f14e00c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e00cfe40 0x7f14e00c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 -- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e00b8440 msgr2=0x7f14e00c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:51.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e00b8440 0x7f14e00c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:51.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14deffd700 1 -- 172.21.15.27:0/485503249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80094a0 con 0x7f14e00b2180 2023-07-06T20:15:51.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14df7fe700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14e00cfe40 0x7f14e00c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:51.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.981+0000 7f14d7fff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14e00b8440 0x7f14e00c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:15:51.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.982+0000 7f14deffd700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00b2180 0x7f14e0149220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c700 tx=0x7f14d000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.982+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0005d70 con 0x7f14e00b2180 2023-07-06T20:15:51.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.982+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d00070d0 con 0x7f14e00b2180 2023-07-06T20:15:51.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.982+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e00cc2a0 con 0x7f14e00b2180 2023-07-06T20:15:51.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.982+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0007460 con 0x7f14e00b2180 2023-07-06T20:15:51.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.982+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e015cee0 con 0x7f14e00b2180 2023-07-06T20:15:51.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.983+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f14e00b8810 con 0x7f14e00b2180 2023-07-06T20:15:51.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.984+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14d00075c0 con 0x7f14e00b2180 2023-07-06T20:15:51.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.985+0000 7f14dcff9700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f14c006ace0 0x7f14c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.985+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f14d0088810 con 0x7f14e00b2180 2023-07-06T20:15:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.985+0000 7f14dcff9700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f14c0071150 0x7f14c0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.985+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 --> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14c0073c30 con 0x7f14c0071150 2023-07-06T20:15:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.985+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d008adf0 con 0x7f14e00b2180 2023-07-06T20:15:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.986+0000 7f14d7fff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f14c006ace0 0x7f14c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.986+0000 7f14df7fe700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f14c0071150 0x7f14c0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.986+0000 7f14d7fff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f14c006ace0 0x7f14c006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14e00d02a0 tx=0x7f14c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.987+0000 7f14df7fe700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f14c0071150 0x7f14c0073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:51.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:51.988+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== osd.5 v2:172.21.15.153:6806/1975714118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f14c0073c30 con 0x7f14c0071150 2023-07-06T20:15:52.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.003+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 --> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f14e00cf3c0 con 0x7f14c0071150 2023-07-06T20:15:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.003+0000 7f14dcff9700 1 -- 172.21.15.27:0/485503249 <== osd.5 v2:172.21.15.153:6806/1975714118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f14d8012380 con 0x7f14c0071150 2023-07-06T20:15:52.006 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:52.006 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:52.006 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f14c0071150 msgr2=0x7f14c0073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6806/1975714118,v1:172.21.15.153:6808/1975714118] conn(0x7f14c0071150 0x7f14c0073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f14c006ace0 msgr2=0x7f14c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f14c006ace0 0x7f14c006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14e00d02a0 tx=0x7f14c8008030 comp rx=0 tx=0).stop 2023-07-06T20:15:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00b2180 msgr2=0x7f14e0149220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 --2- 172.21.15.27:0/485503249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e00b2180 0x7f14e0149220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c700 tx=0x7f14d000cac0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14df7fe700 1 -- 172.21.15.27:0/485503249 reap_dead start 2023-07-06T20:15:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 shutdown_connections 2023-07-06T20:15:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 >> 172.21.15.27:0/485503249 conn(0x7f14e006be10 msgr2=0x7f14e00b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 shutdown_connections 2023-07-06T20:15:52.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.004+0000 7f14e59ff700 1 -- 172.21.15.27:0/485503249 wait complete. 2023-07-06T20:15:52.023 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-06T20:15:52.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.226+0000 7fe5251ff700 1 Processor -- start 2023-07-06T20:15:52.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.226+0000 7fe5251ff700 1 -- start start 2023-07-06T20:15:52.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.227+0000 7fe5251ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe52015fca0 0x7fe5200d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.227+0000 7fe5251ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe5200b6110 0x7fe5200cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.227+0000 7fe5251ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200cf780 con 0x7fe5200b5c90 2023-07-06T20:15:52.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.227+0000 7fe5251ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200cd490 con 0x7fe52015fca0 2023-07-06T20:15:52.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.227+0000 7fe5251ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200cd770 con 0x7fe5200b6110 2023-07-06T20:15:52.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe52015fca0 0x7fe5200d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe52015fca0 0x7fe5200d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45848/0 (socket says 172.21.15.27:45848) 2023-07-06T20:15:52.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 -- 172.21.15.27:0/2222831369 learned_addr learned my addr 172.21.15.27:0/2222831369 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:52.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51f59e700 1 --2- 172.21.15.27:0/2222831369 >> v2:172.21.15.153:3300/0 conn(0x7fe5200b6110 0x7fe5200cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 -- 172.21.15.27:0/2222831369 >> v2:172.21.15.153:3300/0 conn(0x7fe5200b6110 msgr2=0x7fe5200cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 --2- 172.21.15.27:0/2222831369 >> v2:172.21.15.153:3300/0 conn(0x7fe5200b6110 0x7fe5200cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 -- 172.21.15.27:0/2222831369 >> v1:172.21.15.153:6789/0 conn(0x7fe5200b5c90 legacy=0x7fe5200d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 -- 172.21.15.27:0/2222831369 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200cda50 con 0x7fe52015fca0 2023-07-06T20:15:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.228+0000 7fe51ed9d700 1 --2- 172.21.15.27:0/2222831369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52015fca0 0x7fe5200d1420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe5200cd410 tx=0x7fe510009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7e7b2d4b9b65b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe51dd9b700 1 -- 172.21.15.27:0/2222831369 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000b870 con 0x7fe52015fca0 2023-07-06T20:15:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe51dd9b700 1 -- 172.21.15.27:0/2222831369 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51000b9d0 con 0x7fe52015fca0 2023-07-06T20:15:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe51dd9b700 1 -- 172.21.15.27:0/2222831369 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000bd60 con 0x7fe52015fca0 2023-07-06T20:15:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 -- 172.21.15.27:0/2222831369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52015fca0 msgr2=0x7fe5200d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 --2- 172.21.15.27:0/2222831369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52015fca0 0x7fe5200d1420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe5200cd410 tx=0x7fe510009bd0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 -- 172.21.15.27:0/2222831369 shutdown_connections 2023-07-06T20:15:52.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 --2- 172.21.15.27:0/2222831369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52015fca0 0x7fe5200d1420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 --2- 172.21.15.27:0/2222831369 >> v2:172.21.15.153:3300/0 conn(0x7fe5200b6110 0x7fe5200cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 -- 172.21.15.27:0/2222831369 >> 172.21.15.27:0/2222831369 conn(0x7fe52006c150 msgr2=0x7fe52006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:52.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 -- 172.21.15.27:0/2222831369 shutdown_connections 2023-07-06T20:15:52.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.229+0000 7fe5251ff700 1 -- 172.21.15.27:0/2222831369 wait complete. 2023-07-06T20:15:52.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.231+0000 7fe5251ff700 1 Processor -- start 2023-07-06T20:15:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.231+0000 7fe5251ff700 1 -- start start 2023-07-06T20:15:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe5251ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5200b5c90 0x7fe520148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe5251ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 0x7fe5200c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe5251ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe52015fca0 0x7fe5200c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe5251ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe520148c50 con 0x7fe5200b6110 2023-07-06T20:15:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe5251ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52014b230 con 0x7fe52015fca0 2023-07-06T20:15:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe5251ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52014b510 con 0x7fe5200b5c90 2023-07-06T20:15:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.232+0000 7fe51e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 0x7fe5200c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 0x7fe5200c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45864/0 (socket says 172.21.15.27:45864) 2023-07-06T20:15:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 -- 172.21.15.27:0/3203306511 learned_addr learned my addr 172.21.15.27:0/3203306511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 -- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5200b5c90 msgr2=0x7fe520148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5200b5c90 0x7fe520148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 -- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe52015fca0 msgr2=0x7fe5200c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe52015fca0 0x7fe5200c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 -- 172.21.15.27:0/3203306511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009550 con 0x7fe5200b6110 2023-07-06T20:15:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe51e59c700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 0x7fe5200c6a30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c4e0 tx=0x7fe51400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140052d0 con 0x7fe5200b6110 2023-07-06T20:15:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514005430 con 0x7fe5200b6110 2023-07-06T20:15:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52014b7f0 con 0x7fe5200b6110 2023-07-06T20:15:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.233+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140038c0 con 0x7fe5200b6110 2023-07-06T20:15:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.234+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52014bdb0 con 0x7fe5200b6110 2023-07-06T20:15:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.234+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200008d0 con 0x7fe5200b6110 2023-07-06T20:15:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.236+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe514003a20 con 0x7fe5200b6110 2023-07-06T20:15:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.237+0000 7fe5077fe700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe50c06ace0 0x7fe50c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.237+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe514088710 con 0x7fe5200b6110 2023-07-06T20:15:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.237+0000 7fe51ed9d700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe50c06ace0 0x7fe50c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.238+0000 7fe51ed9d700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe50c06ace0 0x7fe50c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5200d02c0 tx=0x7fe510010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.241+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe514053a40 con 0x7fe5200b6110 2023-07-06T20:15:52.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.445+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe52006ae40 con 0x7fe50c06ace0 2023-07-06T20:15:52.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.446+0000 7fe5077fe700 1 -- 172.21.15.27:0/3203306511 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe51000b770 con 0x7fe50c06ace0 2023-07-06T20:15:52.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.448+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe50c06ace0 msgr2=0x7fe50c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.448+0000 7fe5251ff700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe50c06ace0 0x7fe50c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5200d02c0 tx=0x7fe510010000 comp rx=0 tx=0).stop 2023-07-06T20:15:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.448+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 msgr2=0x7fe5200c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.448+0000 7fe5251ff700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 0x7fe5200c6a30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c4e0 tx=0x7fe51400c8a0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 shutdown_connections 2023-07-06T20:15:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fe50c06ace0 0x7fe50c06d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe52015fca0 0x7fe5200c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe5200b5c90 0x7fe520148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 --2- 172.21.15.27:0/3203306511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5200b6110 0x7fe5200c6a30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 >> 172.21.15.27:0/3203306511 conn(0x7fe52006c150 msgr2=0x7fe5200b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 shutdown_connections 2023-07-06T20:15:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.449+0000 7fe5251ff700 1 -- 172.21.15.27:0/3203306511 wait complete. 2023-07-06T20:15:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2023-07-06T20:15:52.469 INFO:tasks.ceph:Scrubbing osd.6 2023-07-06T20:15:52.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.674+0000 7f8927dff700 1 Processor -- start 2023-07-06T20:15:52.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.674+0000 7f8927dff700 1 -- start start 2023-07-06T20:15:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.675+0000 7f8927dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f89200b2180 0x7f89200cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.675+0000 7f8927dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8920160cc0 con 0x7f89200cfe40 2023-07-06T20:15:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.675+0000 7f8927dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8920160f80 con 0x7f89200b8440 2023-07-06T20:15:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.675+0000 7f8927dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8920160ae0 con 0x7f89200b2180 2023-07-06T20:15:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.675+0000 7f892539a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f89200cfe40 0x7f89200d0960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50264/0 (socket says 172.21.15.27:50264) 2023-07-06T20:15:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.675+0000 7f892539a700 1 -- 172.21.15.27:0/3045479463 learned_addr learned my addr 172.21.15.27:0/3045479463 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f892639c700 1 --2- 172.21.15.27:0/3045479463 >> v2:172.21.15.153:3300/0 conn(0x7f89200b2180 0x7f89200cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747262179 0 0) 0x7f8920160cc0 con 0x7f89200cfe40 2023-07-06T20:15:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f890c0035b0 con 0x7f89200cfe40 2023-07-06T20:15:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98590384 0 0) 0x7f8920160f80 con 0x7f89200b8440 2023-07-06T20:15:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8920160cc0 con 0x7f89200b8440 2023-07-06T20:15:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f892639c700 1 -- 172.21.15.27:0/3045479463 >> v1:172.21.15.153:6789/0 conn(0x7f89200b8440 legacy=0x7f89200cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f892639c700 1 -- 172.21.15.27:0/3045479463 >> v1:172.21.15.27:6789/0 conn(0x7f89200cfe40 legacy=0x7f89200d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.676+0000 7f892639c700 1 -- 172.21.15.27:0/3045479463 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89200cf0e0 con 0x7f89200b2180 2023-07-06T20:15:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.677+0000 7f892639c700 1 --2- 172.21.15.27:0/3045479463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 0x7f89200cd530 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8918009d90 tx=0x7f891800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afcd0e1ebf9cdba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.677+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800e070 con 0x7f89200b2180 2023-07-06T20:15:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.677+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891800bb20 con 0x7f89200b2180 2023-07-06T20:15:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.677+0000 7f8924b99700 1 -- 172.21.15.27:0/3045479463 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800be90 con 0x7f89200b2180 2023-07-06T20:15:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 -- 172.21.15.27:0/3045479463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 msgr2=0x7f89200cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 --2- 172.21.15.27:0/3045479463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 0x7f89200cd530 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8918009d90 tx=0x7f891800b330 comp rx=0 tx=0).stop 2023-07-06T20:15:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 -- 172.21.15.27:0/3045479463 shutdown_connections 2023-07-06T20:15:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 --2- 172.21.15.27:0/3045479463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 0x7f89200cd530 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 -- 172.21.15.27:0/3045479463 >> 172.21.15.27:0/3045479463 conn(0x7f892006be10 msgr2=0x7f8920071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 -- 172.21.15.27:0/3045479463 shutdown_connections 2023-07-06T20:15:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.678+0000 7f8927dff700 1 -- 172.21.15.27:0/3045479463 wait complete. 2023-07-06T20:15:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.679+0000 7f8927dff700 1 Processor -- start 2023-07-06T20:15:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.680+0000 7f8927dff700 1 -- start start 2023-07-06T20:15:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.680+0000 7f8927dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 0x7f892015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f8927dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89200b8440 0x7f89200c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f8927dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89200cfe40 0x7f89200c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f8927dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89200c7990 con 0x7f89200b8440 2023-07-06T20:15:52.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f8927dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892015f380 con 0x7f89200b2180 2023-07-06T20:15:52.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f8927dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892015fb50 con 0x7f89200cfe40 2023-07-06T20:15:52.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89200b8440 0x7f89200c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89200b8440 0x7f89200c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45870/0 (socket says 172.21.15.27:45870) 2023-07-06T20:15:52.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 -- 172.21.15.27:0/2266643525 learned_addr learned my addr 172.21.15.27:0/2266643525 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:52.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892639c700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89200cfe40 0x7f89200c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f8925b9b700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 0x7f892015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 -- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89200cfe40 msgr2=0x7f89200c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89200cfe40 0x7f89200c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 -- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 msgr2=0x7f892015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89200b2180 0x7f892015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.681+0000 7f892539a700 1 -- 172.21.15.27:0/2266643525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009a70 con 0x7f89200b8440 2023-07-06T20:15:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.682+0000 7f892539a700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89200b8440 0x7f89200c6860 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8910005910 tx=0x7f891000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.682+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910005d70 con 0x7f89200b8440 2023-07-06T20:15:52.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.682+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000ce80 con 0x7f89200b8440 2023-07-06T20:15:52.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.682+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100072f0 con 0x7f89200b8440 2023-07-06T20:15:52.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.682+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892015f590 con 0x7f89200b8440 2023-07-06T20:15:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.682+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892015e680 con 0x7f89200b8440 2023-07-06T20:15:52.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.683+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89200b8810 con 0x7f89200b8440 2023-07-06T20:15:52.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.684+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8910007450 con 0x7f89200b8440 2023-07-06T20:15:52.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.685+0000 7f89167fc700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f890406ad10 0x7f890406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.685+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8910088590 con 0x7f89200b8440 2023-07-06T20:15:52.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.685+0000 7f89167fc700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f8904071180 0x7f8904073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.685+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 --> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8904073c60 con 0x7f8904071180 2023-07-06T20:15:52.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.685+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f891008ab70 con 0x7f89200b8440 2023-07-06T20:15:52.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.686+0000 7f8925b9b700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f890406ad10 0x7f890406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.686+0000 7f892639c700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f8904071180 0x7f8904073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.687+0000 7f8925b9b700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f890406ad10 0x7f890406d1c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f891c0011e0 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.687+0000 7f892639c700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f8904071180 0x7f8904073590 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.687+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== osd.6 v2:172.21.15.153:6824/2351180975 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8904073c60 con 0x7f8904071180 2023-07-06T20:15:52.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.703+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 --> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f89200cf3c0 con 0x7f8904071180 2023-07-06T20:15:52.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.703+0000 7f89167fc700 1 -- 172.21.15.27:0/2266643525 <== osd.6 v2:172.21.15.153:6824/2351180975 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f89180098f0 con 0x7f8904071180 2023-07-06T20:15:52.705 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:52.706 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:52.706 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f8904071180 msgr2=0x7f8904073590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6824/2351180975,v1:172.21.15.153:6825/2351180975] conn(0x7f8904071180 0x7f8904073590 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f890406ad10 msgr2=0x7f890406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f890406ad10 0x7f890406d1c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f891c0011e0 tx=0x7f891c009040 comp rx=0 tx=0).stop 2023-07-06T20:15:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89200b8440 msgr2=0x7f89200c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 --2- 172.21.15.27:0/2266643525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89200b8440 0x7f89200c6860 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8910005910 tx=0x7f891000c7b0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f892639c700 1 -- 172.21.15.27:0/2266643525 reap_dead start 2023-07-06T20:15:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 shutdown_connections 2023-07-06T20:15:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 >> 172.21.15.27:0/2266643525 conn(0x7f892006be10 msgr2=0x7f89200b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:52.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 shutdown_connections 2023-07-06T20:15:52.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.704+0000 7f8927dff700 1 -- 172.21.15.27:0/2266643525 wait complete. 2023-07-06T20:15:52.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-06T20:15:52.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.926+0000 7fb17a1ff700 1 Processor -- start 2023-07-06T20:15:52.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.926+0000 7fb17a1ff700 1 -- start start 2023-07-06T20:15:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb17a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb17415fca0 0x7fb1740d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb17a1ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1740b6100 0x7fb1740cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb17a1ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740cf780 con 0x7fb1740b5c90 2023-07-06T20:15:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb17a1ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740cd490 con 0x7fb1740b6100 2023-07-06T20:15:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb17a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740cd770 con 0x7fb17415fca0 2023-07-06T20:15:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb173fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1740b6100 0x7fb1740cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb173fff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1740b6100 0x7fb1740cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45876/0 (socket says 172.21.15.27:45876) 2023-07-06T20:15:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb173fff700 1 -- 172.21.15.27:0/46274417 learned_addr learned my addr 172.21.15.27:0/46274417 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.928+0000 7fb1737fe700 1 --2- 172.21.15.27:0/46274417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb17415fca0 0x7fb1740d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb173fff700 1 -- 172.21.15.27:0/46274417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb17415fca0 msgr2=0x7fb1740d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:52.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb173fff700 1 --2- 172.21.15.27:0/46274417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb17415fca0 0x7fb1740d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb173fff700 1 -- 172.21.15.27:0/46274417 >> v1:172.21.15.153:6789/0 conn(0x7fb1740b5c90 legacy=0x7fb1740d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb173fff700 1 -- 172.21.15.27:0/46274417 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740cda50 con 0x7fb1740b6100 2023-07-06T20:15:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb173fff700 1 --2- 172.21.15.27:0/46274417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb1740cfc10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009e50 tx=0x7fb16c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866b20e42cce8354 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb1727fc700 1 -- 172.21.15.27:0/46274417 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00e070 con 0x7fb1740b6100 2023-07-06T20:15:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb1727fc700 1 -- 172.21.15.27:0/46274417 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c00b910 con 0x7fb1740b6100 2023-07-06T20:15:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.929+0000 7fb1727fc700 1 -- 172.21.15.27:0/46274417 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00bcc0 con 0x7fb1740b6100 2023-07-06T20:15:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 -- 172.21.15.27:0/46274417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 msgr2=0x7fb1740cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/46274417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb1740cfc10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009e50 tx=0x7fb16c00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 -- 172.21.15.27:0/46274417 shutdown_connections 2023-07-06T20:15:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/46274417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb1740cfc10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/46274417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb17415fca0 0x7fb1740d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 -- 172.21.15.27:0/46274417 >> 172.21.15.27:0/46274417 conn(0x7fb17406c150 msgr2=0x7fb17406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 -- 172.21.15.27:0/46274417 shutdown_connections 2023-07-06T20:15:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.930+0000 7fb17a1ff700 1 -- 172.21.15.27:0/46274417 wait complete. 2023-07-06T20:15:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.932+0000 7fb17a1ff700 1 Processor -- start 2023-07-06T20:15:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.932+0000 7fb17a1ff700 1 -- start start 2023-07-06T20:15:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb17a1ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1740b5c90 0x7fb1740c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb17a1ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb17415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb17a1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17415fca0 0x7fb1740cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb17a1ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740c9210 con 0x7fb1740b6100 2023-07-06T20:15:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb17a1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17415bbe0 con 0x7fb17415fca0 2023-07-06T20:15:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb17a1ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17415db80 con 0x7fb1740b5c90 2023-07-06T20:15:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb172ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb17415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb172ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb17415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45890/0 (socket says 172.21.15.27:45890) 2023-07-06T20:15:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb172ffd700 1 -- 172.21.15.27:0/2647934087 learned_addr learned my addr 172.21.15.27:0/2647934087 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb1737fe700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1740b5c90 0x7fb1740c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.933+0000 7fb173fff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17415fca0 0x7fb1740cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb172ffd700 1 -- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1740b5c90 msgr2=0x7fb1740c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb172ffd700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1740b5c90 0x7fb1740c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb172ffd700 1 -- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17415fca0 msgr2=0x7fb1740cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb172ffd700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17415fca0 0x7fb1740cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb172ffd700 1 -- 172.21.15.27:0/2647934087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c009b30 con 0x7fb1740b6100 2023-07-06T20:15:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb173fff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17415fca0 0x7fb1740cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb172ffd700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb17415e010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c4e0 tx=0x7fb16800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680052d0 con 0x7fb1740b6100 2023-07-06T20:15:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb168005430 con 0x7fb1740b6100 2023-07-06T20:15:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1680038c0 con 0x7fb1740b6100 2023-07-06T20:15:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1741486c0 con 0x7fb1740b6100 2023-07-06T20:15:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.934+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17415d530 con 0x7fb1740b6100 2023-07-06T20:15:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.935+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17415d390 con 0x7fb1740b6100 2023-07-06T20:15:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.936+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb168003a20 con 0x7fb1740b6100 2023-07-06T20:15:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.937+0000 7fb15bfff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb16006ace0 0x7fb16006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.937+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb168088650 con 0x7fb1740b6100 2023-07-06T20:15:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.938+0000 7fb1737fe700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb16006ace0 0x7fb16006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.938+0000 7fb1737fe700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb16006ace0 0x7fb16006d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb164009ad0 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:52.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:52.942+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb17415d390 con 0x7fb1740b6100 2023-07-06T20:15:53.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.151+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb17406ae40 con 0x7fb16006ace0 2023-07-06T20:15:53.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.152+0000 7fb15bfff700 1 -- 172.21.15.27:0/2647934087 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb164011750 con 0x7fb16006ace0 2023-07-06T20:15:53.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.154+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb16006ace0 msgr2=0x7fb16006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.154+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb16006ace0 0x7fb16006d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb164009ad0 tx=0x7fb164008040 comp rx=0 tx=0).stop 2023-07-06T20:15:53.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.154+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 msgr2=0x7fb17415e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.154+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb17415e010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c4e0 tx=0x7fb16800c8a0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 shutdown_connections 2023-07-06T20:15:53.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb16006ace0 0x7fb16006d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb17415fca0 0x7fb1740cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb1740b5c90 0x7fb1740c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 --2- 172.21.15.27:0/2647934087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1740b6100 0x7fb17415e010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 >> 172.21.15.27:0/2647934087 conn(0x7fb17406c150 msgr2=0x7fb1740b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 shutdown_connections 2023-07-06T20:15:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.155+0000 7fb17a1ff700 1 -- 172.21.15.27:0/2647934087 wait complete. 2023-07-06T20:15:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 6 to deep-scrub 2023-07-06T20:15:53.174 INFO:tasks.ceph:Scrubbing osd.7 2023-07-06T20:15:53.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-06T20:15:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.380+0000 7f8f63dff700 1 Processor -- start 2023-07-06T20:15:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.380+0000 7f8f63dff700 1 -- start start 2023-07-06T20:15:53.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.381+0000 7f8f63dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b4320 0x7f8f5c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.381+0000 7f8f63dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8f5c0b5c50 0x7f8f5c0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.381+0000 7f8f63dff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5c0ce960 con 0x7f8f5c0cf880 2023-07-06T20:15:53.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.381+0000 7f8f63dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5c0b6610 con 0x7f8f5c0b5c50 2023-07-06T20:15:53.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.381+0000 7f8f63dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0b46f0 con 0x7f8f5c0b4320 2023-07-06T20:15:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f61b9b700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b4320 0x7f8f5c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f6139a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8f5c0cf880 0x7f8f5c0cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.27:38208/0 (socket says 172.21.15.27:38208) 2023-07-06T20:15:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f6139a700 1 -- 172.21.15.27:0/2950684984 learned_addr learned my addr 172.21.15.27:0/2950684984 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f6239c700 1 --2- 172.21.15.27:0/2950684984 >> v2:172.21.15.153:3300/0 conn(0x7f8f5c0b5c50 0x7f8f5c0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f60b99700 1 -- 172.21.15.27:0/2950684984 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777064461 0 0) 0x7f8f5c0ce960 con 0x7f8f5c0cf880 2023-07-06T20:15:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f60b99700 1 -- 172.21.15.27:0/2950684984 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f480035b0 con 0x7f8f5c0cf880 2023-07-06T20:15:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f61b9b700 1 -- 172.21.15.27:0/2950684984 >> v2:172.21.15.153:3300/0 conn(0x7f8f5c0b5c50 msgr2=0x7f8f5c0b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f61b9b700 1 --2- 172.21.15.27:0/2950684984 >> v2:172.21.15.153:3300/0 conn(0x7f8f5c0b5c50 0x7f8f5c0b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f61b9b700 1 -- 172.21.15.27:0/2950684984 >> v1:172.21.15.153:6789/0 conn(0x7f8f5c0cf880 legacy=0x7f8f5c0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.382+0000 7f8f61b9b700 1 -- 172.21.15.27:0/2950684984 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0cdd50 con 0x7f8f5c0b4320 2023-07-06T20:15:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.383+0000 7f8f61b9b700 1 --2- 172.21.15.27:0/2950684984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b4320 0x7f8f5c0cd450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009820 tx=0x7f8f58009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0b62bd3ad4a15e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.383+0000 7f8f60b99700 1 -- 172.21.15.27:0/2950684984 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58005e30 con 0x7f8f5c0b4320 2023-07-06T20:15:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.383+0000 7f8f60b99700 1 -- 172.21.15.27:0/2950684984 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f58006030 con 0x7f8f5c0b4320 2023-07-06T20:15:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.383+0000 7f8f60b99700 1 -- 172.21.15.27:0/2950684984 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f580063c0 con 0x7f8f5c0b4320 2023-07-06T20:15:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.383+0000 7f8f63dff700 1 -- 172.21.15.27:0/2950684984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b4320 msgr2=0x7f8f5c0cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.383+0000 7f8f63dff700 1 --2- 172.21.15.27:0/2950684984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b4320 0x7f8f5c0cd450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009820 tx=0x7f8f58009b30 comp rx=0 tx=0).stop 2023-07-06T20:15:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.384+0000 7f8f63dff700 1 -- 172.21.15.27:0/2950684984 shutdown_connections 2023-07-06T20:15:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.384+0000 7f8f63dff700 1 --2- 172.21.15.27:0/2950684984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b4320 0x7f8f5c0cd450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.384+0000 7f8f63dff700 1 --2- 172.21.15.27:0/2950684984 >> v2:172.21.15.153:3300/0 conn(0x7f8f5c0b5c50 0x7f8f5c0b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.384+0000 7f8f63dff700 1 -- 172.21.15.27:0/2950684984 >> 172.21.15.27:0/2950684984 conn(0x7f8f5c09c130 msgr2=0x7f8f5c0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.384+0000 7f8f63dff700 1 -- 172.21.15.27:0/2950684984 shutdown_connections 2023-07-06T20:15:53.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.384+0000 7f8f63dff700 1 -- 172.21.15.27:0/2950684984 wait complete. 2023-07-06T20:15:53.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.385+0000 7f8f63dff700 1 Processor -- start 2023-07-06T20:15:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 -- start start 2023-07-06T20:15:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f5c0b4320 0x7f8f5c15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b5c50 0x7f8f5c1e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f5c0cf880 0x7f8f5c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0cad30 con 0x7f8f5c0cf880 2023-07-06T20:15:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0cb010 con 0x7f8f5c0b4320 2023-07-06T20:15:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.386+0000 7f8f63dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0cb2f0 con 0x7f8f5c0b5c50 2023-07-06T20:15:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f5c0cf880 0x7f8f5c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f5c0cf880 0x7f8f5c0c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45900/0 (socket says 172.21.15.27:45900) 2023-07-06T20:15:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 -- 172.21.15.27:0/1545560999 learned_addr learned my addr 172.21.15.27:0/1545560999 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f61b9b700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f5c0b4320 0x7f8f5c15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6139a700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b5c50 0x7f8f5c1e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 -- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b5c50 msgr2=0x7f8f5c1e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8f5c0b5c50 0x7f8f5c1e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 -- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f5c0b4320 msgr2=0x7f8f5c15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f5c0b4320 0x7f8f5c15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.387+0000 7f8f6239c700 1 -- 172.21.15.27:0/1545560999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009430 con 0x7f8f5c0cf880 2023-07-06T20:15:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.388+0000 7f8f6239c700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f5c0cf880 0x7f8f5c0c8780 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400efc0 tx=0x7f8f5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.388+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54009ea0 con 0x7f8f5c0cf880 2023-07-06T20:15:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.388+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5400f040 con 0x7f8f5c0cf880 2023-07-06T20:15:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.388+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400ced0 con 0x7f8f5c0cf880 2023-07-06T20:15:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.388+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c0cb5d0 con 0x7f8f5c0cf880 2023-07-06T20:15:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.388+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c15d390 con 0x7f8f5c0cf880 2023-07-06T20:15:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.389+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f5c0ce310 con 0x7f8f5c0cf880 2023-07-06T20:15:53.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.390+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f54014480 con 0x7f8f5c0cf880 2023-07-06T20:15:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.391+0000 7f8f527fc700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f4006ace0 0x7f8f4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.391+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8f54089cf0 con 0x7f8f5c0cf880 2023-07-06T20:15:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.391+0000 7f8f527fc700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7f8f40070fe0 0x7f8f400733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.391+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 --> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f40073ac0 con 0x7f8f40070fe0 2023-07-06T20:15:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.391+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f5408c250 con 0x7f8f5c0cf880 2023-07-06T20:15:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.392+0000 7f8f61b9b700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f4006ace0 0x7f8f4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.392+0000 7f8f6139a700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7f8f40070fe0 0x7f8f400733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.392+0000 7f8f6139a700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7f8f40070fe0 0x7f8f400733f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.392+0000 7f8f61b9b700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f4006ace0 0x7f8f4006d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009b70 tx=0x7f8f58008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.393+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== osd.7 v2:172.21.15.153:6816/469437508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8f40073ac0 con 0x7f8f40070fe0 2023-07-06T20:15:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.408+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 --> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8f5c0ce030 con 0x7f8f40070fe0 2023-07-06T20:15:53.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.409+0000 7f8f527fc700 1 -- 172.21.15.27:0/1545560999 <== osd.7 v2:172.21.15.153:6816/469437508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8f4c004870 con 0x7f8f40070fe0 2023-07-06T20:15:53.411 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-07-06T20:15:53.411 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T20:15:53.412 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-07-06T20:15:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.409+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7f8f40070fe0 msgr2=0x7f8f400733f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.409+0000 7f8f63dff700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6816/469437508,v1:172.21.15.153:6817/469437508] conn(0x7f8f40070fe0 0x7f8f400733f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f4006ace0 msgr2=0x7f8f4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f8f4006ace0 0x7f8f4006d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009b70 tx=0x7f8f58008070 comp rx=0 tx=0).stop 2023-07-06T20:15:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f5c0cf880 msgr2=0x7f8f5c0c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 --2- 172.21.15.27:0/1545560999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f5c0cf880 0x7f8f5c0c8780 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400efc0 tx=0x7f8f5400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f6239c700 1 -- 172.21.15.27:0/1545560999 reap_dead start 2023-07-06T20:15:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 shutdown_connections 2023-07-06T20:15:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 >> 172.21.15.27:0/1545560999 conn(0x7f8f5c09c130 msgr2=0x7f8f5c0b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 shutdown_connections 2023-07-06T20:15:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.410+0000 7f8f63dff700 1 -- 172.21.15.27:0/1545560999 wait complete. 2023-07-06T20:15:53.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-06T20:15:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.631+0000 7f4d17bff700 1 Processor -- start 2023-07-06T20:15:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.631+0000 7f4d17bff700 1 -- start start 2023-07-06T20:15:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.632+0000 7f4d17bff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d100adba0 0x7f4d100cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.632+0000 7f4d17bff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100cd370 con 0x7f4d10158cd0 2023-07-06T20:15:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.632+0000 7f4d17bff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100ad7f0 con 0x7f4d100c9010 2023-07-06T20:15:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.632+0000 7f4d17bff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100cbf50 con 0x7f4d100adba0 2023-07-06T20:15:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d1599b700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4d10158cd0 0x7f4d100cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50276/0 (socket says 172.21.15.27:50276) 2023-07-06T20:15:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d1599b700 1 -- 172.21.15.27:0/528908409 learned_addr learned my addr 172.21.15.27:0/528908409 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d1619c700 1 --2- 172.21.15.27:0/528908409 >> v2:172.21.15.27:3300/0 conn(0x7f4d100adba0 0x7f4d100cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267233141 0 0) 0x7f4d100cd370 con 0x7f4d10158cd0 2023-07-06T20:15:53.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cfc0034d0 con 0x7f4d10158cd0 2023-07-06T20:15:53.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585659054 0 0) 0x7f4d100ad7f0 con 0x7f4d100c9010 2023-07-06T20:15:53.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d100cd370 con 0x7f4d100c9010 2023-07-06T20:15:53.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d1619c700 1 -- 172.21.15.27:0/528908409 >> v1:172.21.15.153:6789/0 conn(0x7f4d100c9010 legacy=0x7f4d1006b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d1619c700 1 -- 172.21.15.27:0/528908409 >> v1:172.21.15.27:6789/0 conn(0x7f4d10158cd0 legacy=0x7f4d100cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.633+0000 7f4d1619c700 1 -- 172.21.15.27:0/528908409 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100cc160 con 0x7f4d100adba0 2023-07-06T20:15:53.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d1619c700 1 --2- 172.21.15.27:0/528908409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100adba0 0x7f4d100cc590 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009d90 tx=0x7f4d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a6aa06a3303633 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800e070 con 0x7f4d100adba0 2023-07-06T20:15:53.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800bb20 con 0x7f4d100adba0 2023-07-06T20:15:53.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d14999700 1 -- 172.21.15.27:0/528908409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08012430 con 0x7f4d100adba0 2023-07-06T20:15:53.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d17bff700 1 -- 172.21.15.27:0/528908409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100adba0 msgr2=0x7f4d100cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d17bff700 1 --2- 172.21.15.27:0/528908409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100adba0 0x7f4d100cc590 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009d90 tx=0x7f4d0800b330 comp rx=0 tx=0).stop 2023-07-06T20:15:53.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d17bff700 1 -- 172.21.15.27:0/528908409 shutdown_connections 2023-07-06T20:15:53.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d17bff700 1 --2- 172.21.15.27:0/528908409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100adba0 0x7f4d100cc590 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d17bff700 1 -- 172.21.15.27:0/528908409 >> 172.21.15.27:0/528908409 conn(0x7f4d100cb440 msgr2=0x7f4d100cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:53.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.634+0000 7f4d17bff700 1 -- 172.21.15.27:0/528908409 shutdown_connections 2023-07-06T20:15:53.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.635+0000 7f4d17bff700 1 -- 172.21.15.27:0/528908409 wait complete. 2023-07-06T20:15:53.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.636+0000 7f4d17bff700 1 Processor -- start 2023-07-06T20:15:53.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.636+0000 7f4d17bff700 1 -- start start 2023-07-06T20:15:53.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.637+0000 7f4d17bff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d100adba0 0x7f4d101590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.637+0000 7f4d17bff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 0x7f4d1006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.637+0000 7f4d17bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d10158cd0 0x7f4d10156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.637+0000 7f4d17bff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100c6e30 con 0x7f4d100c9010 2023-07-06T20:15:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.637+0000 7f4d17bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100b28d0 con 0x7f4d10158cd0 2023-07-06T20:15:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.637+0000 7f4d17bff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100c6ae0 con 0x7f4d100adba0 2023-07-06T20:15:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 0x7f4d1006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 0x7f4d1006fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45914/0 (socket says 172.21.15.27:45914) 2023-07-06T20:15:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 -- 172.21.15.27:0/2182016907 learned_addr learned my addr 172.21.15.27:0/2182016907 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1619c700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d10158cd0 0x7f4d10156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1599b700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d100adba0 0x7f4d101590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 -- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d100adba0 msgr2=0x7f4d101590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d100adba0 0x7f4d101590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 -- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d10158cd0 msgr2=0x7f4d10156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d10158cd0 0x7f4d10156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 -- 172.21.15.27:0/2182016907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08009a70 con 0x7f4d100c9010 2023-07-06T20:15:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.638+0000 7f4d1519a700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 0x7f4d1006fbf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000c500 tx=0x7f4d0000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.639+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000052d0 con 0x7f4d100c9010 2023-07-06T20:15:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.639+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00005450 con 0x7f4d100c9010 2023-07-06T20:15:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.639+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000038c0 con 0x7f4d100c9010 2023-07-06T20:15:53.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.639+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d100b2b90 con 0x7f4d100c9010 2023-07-06T20:15:53.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.639+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d100b3150 con 0x7f4d100c9010 2023-07-06T20:15:53.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.640+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100b3310 con 0x7f4d100c9010 2023-07-06T20:15:53.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.642+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d00003a20 con 0x7f4d100c9010 2023-07-06T20:15:53.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.643+0000 7f4d067fc700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4cf406ace0 0x7f4cf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:53.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.643+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4d000888f0 con 0x7f4d100c9010 2023-07-06T20:15:53.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.644+0000 7f4d1599b700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4cf406ace0 0x7f4cf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.644+0000 7f4d1599b700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4cf406ace0 0x7f4cf406d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c007fd0 tx=0x7f4d0c007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.646+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d00053d30 con 0x7f4d100c9010 2023-07-06T20:15:53.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.852+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d100cd9c0 con 0x7f4cf406ace0 2023-07-06T20:15:53.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.852+0000 7f4d067fc700 1 -- 172.21.15.27:0/2182016907 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4d0c0058e0 con 0x7f4cf406ace0 2023-07-06T20:15:53.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4cf406ace0 msgr2=0x7f4cf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4cf406ace0 0x7f4cf406d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c007fd0 tx=0x7f4d0c007dc0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 msgr2=0x7f4d1006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:53.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 0x7f4d1006fbf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000c500 tx=0x7f4d0000c810 comp rx=0 tx=0).stop 2023-07-06T20:15:53.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 shutdown_connections 2023-07-06T20:15:53.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f4cf406ace0 0x7f4cf406d190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d10158cd0 0x7f4d10156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4d100adba0 0x7f4d101590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 --2- 172.21.15.27:0/2182016907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d100c9010 0x7f4d1006fbf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:53.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 >> 172.21.15.27:0/2182016907 conn(0x7f4d100cb440 msgr2=0x7f4d100cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:53.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 shutdown_connections 2023-07-06T20:15:53.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:53.855+0000 7f4d17bff700 1 -- 172.21.15.27:0/2182016907 wait complete. 2023-07-06T20:15:53.859 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 7 to deep-scrub 2023-07-06T20:15:53.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:15:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.077+0000 7f5afc7ff700 1 Processor -- start 2023-07-06T20:15:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.078+0000 7f5afc7ff700 1 -- start start 2023-07-06T20:15:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af415fca0 0x7f5af40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afc7ff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5af40b6100 0x7f5af40cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afc7ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40cf780 con 0x7f5af40b5c90 2023-07-06T20:15:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afc7ff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40cd490 con 0x7f5af40b6100 2023-07-06T20:15:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40cd770 con 0x7f5af415fca0 2023-07-06T20:15:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afad9c700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5af40b6100 0x7f5af40cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5af9d9a700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5af40b5c90 0x7f5af40d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50290/0 (socket says 172.21.15.27:50290) 2023-07-06T20:15:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5af9d9a700 1 -- 172.21.15.27:0/1689544851 learned_addr learned my addr 172.21.15.27:0/1689544851 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:15:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.079+0000 7f5afa59b700 1 --2- 172.21.15.27:0/1689544851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af415fca0 0x7f5af40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5afad9c700 1 -- 172.21.15.27:0/1689544851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af415fca0 msgr2=0x7f5af40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:15:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5afad9c700 1 --2- 172.21.15.27:0/1689544851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af415fca0 0x7f5af40d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5afad9c700 1 -- 172.21.15.27:0/1689544851 >> v1:172.21.15.27:6789/0 conn(0x7f5af40b5c90 legacy=0x7f5af40d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5afad9c700 1 -- 172.21.15.27:0/1689544851 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40cda50 con 0x7f5af40b6100 2023-07-06T20:15:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5afad9c700 1 --2- 172.21.15.27:0/1689544851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af40cfc10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009d90 tx=0x7f5aec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6621317281e7939a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5af9599700 1 -- 172.21.15.27:0/1689544851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00e070 con 0x7f5af40b6100 2023-07-06T20:15:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5af9599700 1 -- 172.21.15.27:0/1689544851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00bbd0 con 0x7f5af40b6100 2023-07-06T20:15:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.080+0000 7f5af9599700 1 -- 172.21.15.27:0/1689544851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec012430 con 0x7f5af40b6100 2023-07-06T20:15:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 -- 172.21.15.27:0/1689544851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 msgr2=0x7f5af40cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/1689544851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af40cfc10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009d90 tx=0x7f5aec00b330 comp rx=0 tx=0).stop 2023-07-06T20:15:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 -- 172.21.15.27:0/1689544851 shutdown_connections 2023-07-06T20:15:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/1689544851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af40cfc10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/1689544851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af415fca0 0x7f5af40d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 -- 172.21.15.27:0/1689544851 >> 172.21.15.27:0/1689544851 conn(0x7f5af406c150 msgr2=0x7f5af406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 -- 172.21.15.27:0/1689544851 shutdown_connections 2023-07-06T20:15:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.081+0000 7f5afc7ff700 1 -- 172.21.15.27:0/1689544851 wait complete. 2023-07-06T20:15:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.083+0000 7f5afc7ff700 1 Processor -- start 2023-07-06T20:15:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.083+0000 7f5afc7ff700 1 -- start start 2023-07-06T20:15:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af40b5c90 0x7f5af40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af415fca0 0x7f5af40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40c9210 con 0x7f5af40b6100 2023-07-06T20:15:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af415bbe0 con 0x7f5af415fca0 2023-07-06T20:15:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af415db80 con 0x7f5af40b5c90 2023-07-06T20:15:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5af9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5af9d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45940/0 (socket says 172.21.15.27:45940) 2023-07-06T20:15:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5af9d9a700 1 -- 172.21.15.27:0/410504090 learned_addr learned my addr 172.21.15.27:0/410504090 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:15:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afa59b700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af40b5c90 0x7f5af40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:54.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.084+0000 7f5afad9c700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af415fca0 0x7f5af40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5af9d9a700 1 -- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af40b5c90 msgr2=0x7f5af40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5af9d9a700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af40b5c90 0x7f5af40c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5af9d9a700 1 -- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af415fca0 msgr2=0x7f5af40cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5af9d9a700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af415fca0 0x7f5af40cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5af9d9a700 1 -- 172.21.15.27:0/410504090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009a70 con 0x7f5af40b6100 2023-07-06T20:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5afad9c700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af415fca0 0x7f5af40cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5af9d9a700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af415e010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c520 tx=0x7f5ae400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4005d70 con 0x7f5af40b6100 2023-07-06T20:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae40070d0 con 0x7f5af40b6100 2023-07-06T20:15:54.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4007440 con 0x7f5af40b6100 2023-07-06T20:15:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af41486c0 con 0x7f5af40b6100 2023-07-06T20:15:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.085+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af415d530 con 0x7f5af40b6100 2023-07-06T20:15:54.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.086+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af415d390 con 0x7f5af40b6100 2023-07-06T20:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.087+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ae40075a0 con 0x7f5af40b6100 2023-07-06T20:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.088+0000 7f5aeaffd700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ad806ace0 0x7f5ad806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:54.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.089+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5ae40884b0 con 0x7f5af40b6100 2023-07-06T20:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.089+0000 7f5afa59b700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ad806ace0 0x7f5ad806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.090+0000 7f5afa59b700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ad806ace0 0x7f5ad806d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5af0006b20 tx=0x7f5af0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:54.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.092+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5af415d390 con 0x7f5af40b6100 2023-07-06T20:15:54.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.297+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5af406ae40 con 0x7f5ad806ace0 2023-07-06T20:15:54.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.300+0000 7f5aeaffd700 1 -- 172.21.15.27:0/410504090 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72871 (secure 0 0 0) 0x7f5af00076b0 con 0x7f5ad806ace0 2023-07-06T20:15:54.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:15:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ad806ace0 msgr2=0x7f5ad806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ad806ace0 0x7f5ad806d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5af0006b20 tx=0x7f5af0008040 comp rx=0 tx=0).stop 2023-07-06T20:15:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 msgr2=0x7f5af415e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af415e010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c520 tx=0x7f5ae400c830 comp rx=0 tx=0).stop 2023-07-06T20:15:54.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 shutdown_connections 2023-07-06T20:15:54.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f5ad806ace0 0x7f5ad806d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af415fca0 0x7f5af40cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5af40b5c90 0x7f5af40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 --2- 172.21.15.27:0/410504090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af40b6100 0x7f5af415e010 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 >> 172.21.15.27:0/410504090 conn(0x7f5af406c150 msgr2=0x7f5af40b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:54.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 shutdown_connections 2023-07-06T20:15:54.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:15:54.303+0000 7f5afc7ff700 1 -- 172.21.15.27:0/410504090 wait complete. 2023-07-06T20:15:54.307 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:15:54.322 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-07-06T20:15:52.369953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220420,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754300,"statfs":{"total":773094113280,"available":772868403200,"internally_reserved":0,"allocated":4919296,"data_stored":2739517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.248227"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630997+0000","last_change":"2023-07-06T20:13:53.341818+0000","last_active":"2023-07-06T20:15:42.630997+0000","last_peered":"2023-07-06T20:15:42.630997+0000","last_clean":"2023-07-06T20:15:42.630997+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:42.630997+0000","last_undegraded":"2023-07-06T20:15:42.630997+0000","last_fullsized":"2023-07-06T20:15:42.630997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:44:21.692262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.632122+0000","last_change":"2023-07-06T20:13:52.981356+0000","last_active":"2023-07-06T20:15:42.632122+0000","last_peered":"2023-07-06T20:15:42.632122+0000","last_clean":"2023-07-06T20:15:42.632122+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:42.632122+0000","last_undegraded":"2023-07-06T20:15:42.632122+0000","last_fullsized":"2023-07-06T20:15:42.632122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:16:31.488104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630991+0000","last_change":"2023-07-06T20:13:53.339441+0000","last_active":"2023-07-06T20:15:42.630991+0000","last_peered":"2023-07-06T20:15:42.630991+0000","last_clean":"2023-07-06T20:15:42.630991+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:42.630991+0000","last_undegraded":"2023-07-06T20:15:42.630991+0000","last_fullsized":"2023-07-06T20:15:42.630991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:28:43.517463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630297+0000","last_change":"2023-07-06T20:13:52.981119+0000","last_active":"2023-07-06T20:15:42.630297+0000","last_peered":"2023-07-06T20:15:42.630297+0000","last_clean":"2023-07-06T20:15:42.630297+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:42.630297+0000","last_undegraded":"2023-07-06T20:15:42.630297+0000","last_fullsized":"2023-07-06T20:15:42.630297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:40:46.329697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.631452+0000","last_change":"2023-07-06T20:13:45.595587+0000","last_active":"2023-07-06T20:15:42.631452+0000","last_peered":"2023-07-06T20:15:42.631452+0000","last_clean":"2023-07-06T20:15:42.631452+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:42.631452+0000","last_undegraded":"2023-07-06T20:15:42.631452+0000","last_fullsized":"2023-07-06T20:15:42.631452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:44.492086+0000","last_clean_scrub_stamp":"2023-07-06T20:13:44.492086+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:45:49.298285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.631433+0000","last_change":"2023-07-06T20:13:52.980626+0000","last_active":"2023-07-06T20:15:42.631433+0000","last_peered":"2023-07-06T20:15:42.631433+0000","last_clean":"2023-07-06T20:15:42.631433+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:15:42.631433+0000","last_undegraded":"2023-07-06T20:15:42.631433+0000","last_fullsized":"2023-07-06T20:15:42.631433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:43.584582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.630552+0000","last_change":"2023-07-06T20:13:53.342050+0000","last_active":"2023-07-06T20:15:42.630552+0000","last_peered":"2023-07-06T20:15:42.630552+0000","last_clean":"2023-07-06T20:15:42.630552+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:42.630552+0000","last_undegraded":"2023-07-06T20:15:42.630552+0000","last_fullsized":"2023-07-06T20:15:42.630552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:16:31.488104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.632088+0000","last_change":"2023-07-06T20:13:52.977437+0000","last_active":"2023-07-06T20:15:42.632088+0000","last_peered":"2023-07-06T20:15:42.632088+0000","last_clean":"2023-07-06T20:15:42.632088+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:42.632088+0000","last_undegraded":"2023-07-06T20:15:42.632088+0000","last_fullsized":"2023-07-06T20:15:42.632088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:44:04.052137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:42.357268+0000","last_change":"2023-07-06T20:13:52.977803+0000","last_active":"2023-07-06T20:15:42.357268+0000","last_peered":"2023-07-06T20:15:42.357268+0000","last_clean":"2023-07-06T20:15:42.357268+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:42.357268+0000","last_undegraded":"2023-07-06T20:15:42.357268+0000","last_fullsized":"2023-07-06T20:15:42.357268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:51.955139+0000","last_clean_scrub_stamp":"2023-07-06T20:13:51.955139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:17:04.184086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":1060864,"data_stored":785343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92200000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.746},"min":{"1min":0.49299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82299999999999995,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85199999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.54700000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.86799999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83799999999999997,"5min":1.776,"15min":1.776},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83599999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67000000000000004,"15min":0.67600000000000005},"min":{"1min":0.44800000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96099999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.089,"15min":1.089},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.51800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91000000000000003,"5min":1.02,"15min":1.02},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.59199999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89100000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.55400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.80800000000000005}]},{"osd":1,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.434,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.752},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.83099999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.077,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.82299999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98699999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.77700000000000002,"15min":0.749},"min":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.125,"15min":1.125},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0149999999999999,"5min":1.375,"15min":1.375},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.436,"5min":0.433,"15min":0.433},"max":{"1min":0.91700000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95799999999999996,"5min":1.181,"15min":1.181},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81200000000000006,"15min":0.81799999999999995},"min":{"1min":0.65100000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95499999999999996,"5min":1.228,"15min":1.228},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.83699999999999997,"15min":0.85099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.196,"15min":1.196},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.316,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66300000000000003,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.52800000000000002,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.56499999999999995,"5min":1.889,"15min":1.889},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66700000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.68799999999999994,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70199999999999996,"5min":2.226,"15min":2.226},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":0.745},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.86599999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95799999999999996,"5min":1.823,"15min":1.823},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86399999999999999,"15min":0.873},"min":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.976,"15min":1.976},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85999999999999999,"15min":0.88},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96799999999999997,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.84699999999999998,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.004,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.873,"15min":0.89200000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0089999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.90100000000000002,"15min":0.92200000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.032,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.73499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":1056768,"data_stored":784324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.502,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.42499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.495,"15min":0.505},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65600000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.55900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.499}]},{"osd":3,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68100000000000005,"5min":1.008,"15min":1.008},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.57799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.871,"5min":1.369,"15min":1.369},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.56000000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.876,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.78100000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.78100000000000003,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.81399999999999995,"15min":0.82899999999999996},"min":{"1min":0.60499999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.874,"5min":1.212,"15min":1.212},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93500000000000005,"5min":1.157,"15min":1.157},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.82399999999999995},"min":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59299999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68999999999999995,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49399999999999999,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69099999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1020000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94899999999999995,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.90300000000000002,"5min":1.258,"15min":1.258},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1499999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.89800000000000002}]},{"osd":6,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95699999999999996,"5min":1.679,"15min":1.679},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.98399999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81599999999999995,"15min":0.82299999999999995},"min":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.089,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.86799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.90400000000000003,"15min":0.98099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60999999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.437,"5min":1.1990000000000001,"15min":1.3260000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61599999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.59099999999999997,"15min":0.61399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.60999999999999999,"5min":2.278,"15min":2.278},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.98599999999999999,"15min":1.0660000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81899999999999995,"5min":11.145,"15min":11.145},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.66000000000000003,"15min":0.68200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":1.0620000000000001,"15min":1.1479999999999999},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82299999999999995,"5min":11.664,"15min":11.664},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.456,"15min":1.5780000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.997,"5min":11.208,"15min":11.208},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":1.1559999999999999,"15min":1.226},"min":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.95199999999999996,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":1.2110000000000001,"15min":1.2909999999999999},"min":{"1min":0.56799999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.89000000000000001,"5min":10.997999999999999,"15min":10.997999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.752,"5min":1.5029999999999999,"15min":1.6279999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98899999999999999,"5min":11.286,"15min":11.286},"last":0.98899999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.93700000000000006},"min":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.2390000000000001,"15min":1.319},"min":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.93700000000000006,"5min":11.055,"15min":11.055},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":1.5069999999999999,"15min":1.629},"min":{"1min":0.65000000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.024,"5min":11.247999999999999,"15min":11.247999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.91800000000000004,"15min":0.94399999999999995},"min":{"1min":0.64100000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.95299999999999996,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.92100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.879}]},{"osd":1,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.0010000000000003,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.7939999999999996,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.99199999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":5.1710000000000003,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":5.1200000000000001,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":1.038}]},{"osd":7,"last update":"Thu Jul 6 20:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.83199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.628},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.505},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66000000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.53700000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.432,"5min":0.373,"15min":0.373},"max":{"1min":0.83199999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.58899999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93799999999999994,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.68700000000000006,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:15:54.327 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.327 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.327 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.328 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.328 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-06T20:13:44.492086+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.328 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.329 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.329 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.329 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-07-06T20:13:51.955139+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=20, tm_min=15, tm_sec=47, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T20:15:54.329 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-06T20:16:14.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:16:14.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.544+0000 7f6ce55ff700 1 Processor -- start 2023-07-06T20:16:14.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.545+0000 7f6ce55ff700 1 -- start start 2023-07-06T20:16:14.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6ce55ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6ce00cc000 0x7f6ce00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6ce55ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 0x7f6ce00ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6ce55ff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce015ca10 con 0x7f6ce00b0440 2023-07-06T20:16:14.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6ce55ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce006c890 con 0x7f6ce00cc000 2023-07-06T20:16:14.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6ce55ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce006b870 con 0x7f6ce006bd00 2023-07-06T20:16:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6cdeffd700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6ce00b0440 0x7f6ce00b0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34062/0 (socket says 172.21.15.27:34062) 2023-07-06T20:16:14.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6cdeffd700 1 -- 172.21.15.27:0/440889078 learned_addr learned my addr 172.21.15.27:0/440889078 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:16:14.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6cdf7fe700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 0x7f6ce00ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.546+0000 7f6cd7fff700 1 --2- 172.21.15.27:0/440889078 >> v2:172.21.15.153:3300/0 conn(0x7f6ce00cc000 0x7f6ce00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321102968 0 0) 0x7f6ce015ca10 con 0x7f6ce00b0440 2023-07-06T20:16:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ccc0035b0 con 0x7f6ce00b0440 2023-07-06T20:16:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089889123 0 0) 0x7f6ccc0035b0 con 0x7f6ce00b0440 2023-07-06T20:16:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 msgr2=0x7f6ce00ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:16:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 0x7f6ce00ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 >> v2:172.21.15.153:3300/0 conn(0x7f6ce00cc000 msgr2=0x7f6ce00cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:16:14.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 --2- 172.21.15.27:0/440889078 >> v2:172.21.15.153:3300/0 conn(0x7f6ce00cc000 0x7f6ce00cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce006cdb0 con 0x7f6ce00b0440 2023-07-06T20:16:14.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 134062271 0 0) 0x7f6cd0001670 con 0x7f6ce00b0440 2023-07-06T20:16:14.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 >> v1:172.21.15.27:6789/0 conn(0x7f6ce00b0440 legacy=0x7f6ce00b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 0x7f6ccc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ccc003fd0 0x7f6ccc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ccc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce006c890 con 0x7f6ccc004d90 2023-07-06T20:16:14.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce006b870 con 0x7f6ccc003fd0 2023-07-06T20:16:14.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc00ba90 con 0x7f6ce006bd00 2023-07-06T20:16:14.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cdeffd700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 0x7f6ccc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cdf7fe700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ccc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.547+0000 7f6cd7fff700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ccc003fd0 0x7f6ccc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cdf7fe700 1 -- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 msgr2=0x7f6ccc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:16:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cdf7fe700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce006bd00 0x7f6ccc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cdf7fe700 1 -- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ccc003fd0 msgr2=0x7f6ccc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:16:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cdf7fe700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ccc003fd0 0x7f6ccc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cdf7fe700 1 -- 172.21.15.27:0/440889078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0035b0 con 0x7f6ccc004d90 2023-07-06T20:16:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cdf7fe700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ccc005200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009fc0 tx=0x7f6ccc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f236461fc9b88b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:14.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800ec30 con 0x7f6ccc004d90 2023-07-06T20:16:14.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd800f040 con 0x7f6ccc004d90 2023-07-06T20:16:14.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.548+0000 7f6cde7fc700 1 -- 172.21.15.27:0/440889078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8014480 con 0x7f6ccc004d90 2023-07-06T20:16:14.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6ce55ff700 1 -- 172.21.15.27:0/440889078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 msgr2=0x7f6ccc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/440889078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ccc005200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009fc0 tx=0x7f6ccc005880 comp rx=0 tx=0).stop 2023-07-06T20:16:14.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6cdf7fe700 1 -- 172.21.15.27:0/440889078 reap_dead start 2023-07-06T20:16:14.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6ce55ff700 1 -- 172.21.15.27:0/440889078 shutdown_connections 2023-07-06T20:16:14.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6ce55ff700 1 -- 172.21.15.27:0/440889078 >> 172.21.15.27:0/440889078 conn(0x7f6ce00cb440 msgr2=0x7f6ce00cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:14.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6ce55ff700 1 -- 172.21.15.27:0/440889078 shutdown_connections 2023-07-06T20:16:14.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.549+0000 7f6ce55ff700 1 -- 172.21.15.27:0/440889078 wait complete. 2023-07-06T20:16:14.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.551+0000 7f6ce55ff700 1 Processor -- start 2023-07-06T20:16:14.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.551+0000 7f6ce55ff700 1 -- start start 2023-07-06T20:16:14.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6ce55ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ce00c2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6ce55ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ce0150590 0x7f6ce015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6ce55ff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce0150a00 0x7f6ce0160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6ce55ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00c25b0 con 0x7f6ccc004d90 2023-07-06T20:16:14.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6ce55ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce015dc80 con 0x7f6ce0150590 2023-07-06T20:16:14.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6ce55ff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce015df20 con 0x7f6ce0150a00 2023-07-06T20:16:14.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ce00c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ce00c2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53986/0 (socket says 172.21.15.27:53986) 2023-07-06T20:16:14.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6cdeffd700 1 -- 172.21.15.27:0/2793509450 learned_addr learned my addr 172.21.15.27:0/2793509450 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:16:14.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6cdf7fe700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce0150a00 0x7f6ce0160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.552+0000 7f6cd7fff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ce0150590 0x7f6ce015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cdeffd700 1 -- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce0150a00 msgr2=0x7f6ce0160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cdeffd700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce0150a00 0x7f6ce0160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cdeffd700 1 -- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ce0150590 msgr2=0x7f6ce015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cdeffd700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ce0150590 0x7f6ce015fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cdeffd700 1 -- 172.21.15.27:0/2793509450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce015e200 con 0x7f6ccc004d90 2023-07-06T20:16:14.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cdeffd700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ce00c2070 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0007310 tx=0x7f6cd00076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:14.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00077f0 con 0x7f6ccc004d90 2023-07-06T20:16:14.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd000f8d0 con 0x7f6ccc004d90 2023-07-06T20:16:14.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000fc80 con 0x7f6ccc004d90 2023-07-06T20:16:14.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce015e4e0 con 0x7f6ccc004d90 2023-07-06T20:16:14.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.553+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce00b2320 con 0x7f6ccc004d90 2023-07-06T20:16:14.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.554+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce015e6d0 con 0x7f6ccc004d90 2023-07-06T20:16:14.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.555+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6cd001d460 con 0x7f6ccc004d90 2023-07-06T20:16:14.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.556+0000 7f6cd77fe700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6cc006ace0 0x7f6cc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.556+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6cd0089370 con 0x7f6ccc004d90 2023-07-06T20:16:14.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.557+0000 7f6cd7fff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6cc006ace0 0x7f6cc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.558+0000 7f6cd7fff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6cc006ace0 0x7f6cc006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009ba0 tx=0x7f6cc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:14.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.561+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6cd00546a0 con 0x7f6ccc004d90 2023-07-06T20:16:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.764+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 --> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ce006b220 con 0x7f6cc006ace0 2023-07-06T20:16:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.767+0000 7f6cd77fe700 1 -- 172.21.15.27:0/2793509450 <== mgr.4108 v2:172.21.15.153:6832/310108246 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72903 (secure 0 0 0) 0x7f6cc80097f0 con 0x7f6cc006ace0 2023-07-06T20:16:14.770 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:16:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6cc006ace0 msgr2=0x7f6cc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6cc006ace0 0x7f6cc006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009ba0 tx=0x7f6cc8008040 comp rx=0 tx=0).stop 2023-07-06T20:16:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 msgr2=0x7f6ce00c2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ce00c2070 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0007310 tx=0x7f6cd00076d0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 shutdown_connections 2023-07-06T20:16:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f6cc006ace0 0x7f6cc006d190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ce0150a00 0x7f6ce0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ccc004d90 0x7f6ce00c2070 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 --2- 172.21.15.27:0/2793509450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ce0150590 0x7f6ce015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 >> 172.21.15.27:0/2793509450 conn(0x7f6ce00cb440 msgr2=0x7f6ce00ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:14.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 shutdown_connections 2023-07-06T20:16:14.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.770+0000 7f6ce55ff700 1 -- 172.21.15.27:0/2793509450 wait complete. 2023-07-06T20:16:14.774 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-07-06T20:16:14.789 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-07-06T20:16:14.375036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220420,"kb_used_data":4804,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754300,"statfs":{"total":773094113280,"available":772868403200,"internally_reserved":0,"allocated":4919296,"data_stored":2739517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:51.485891+0000","last_change":"2023-07-06T20:15:51.485891+0000","last_active":"2023-07-06T20:15:51.485891+0000","last_peered":"2023-07-06T20:15:51.485891+0000","last_clean":"2023-07-06T20:15:51.485891+0000","last_became_active":"2023-07-06T20:13:53.341487+0000","last_became_peered":"2023-07-06T20:13:53.341487+0000","last_unstale":"2023-07-06T20:15:51.485891+0000","last_undegraded":"2023-07-06T20:15:51.485891+0000","last_fullsized":"2023-07-06T20:15:51.485891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:51.485743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:51.485743+0000","last_clean_scrub_stamp":"2023-07-06T20:15:51.485743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:29:33.502944+0000","scrub_duration":0.001465348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:52.705890+0000","last_change":"2023-07-06T20:15:52.705890+0000","last_active":"2023-07-06T20:15:52.705890+0000","last_peered":"2023-07-06T20:15:52.705890+0000","last_clean":"2023-07-06T20:15:52.705890+0000","last_became_active":"2023-07-06T20:13:52.980988+0000","last_became_peered":"2023-07-06T20:13:52.980988+0000","last_unstale":"2023-07-06T20:15:52.705890+0000","last_undegraded":"2023-07-06T20:15:52.705890+0000","last_fullsized":"2023-07-06T20:15:52.705890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:52.705733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:52.705733+0000","last_clean_scrub_stamp":"2023-07-06T20:15:52.705733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:02:10.790219+0000","scrub_duration":0.0018480580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:51.740005+0000","last_change":"2023-07-06T20:15:51.740005+0000","last_active":"2023-07-06T20:15:51.740005+0000","last_peered":"2023-07-06T20:15:51.740005+0000","last_clean":"2023-07-06T20:15:51.740005+0000","last_became_active":"2023-07-06T20:13:53.339119+0000","last_became_peered":"2023-07-06T20:13:53.339119+0000","last_unstale":"2023-07-06T20:15:51.740005+0000","last_undegraded":"2023-07-06T20:15:51.740005+0000","last_fullsized":"2023-07-06T20:15:51.740005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:51.739892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:51.739892+0000","last_clean_scrub_stamp":"2023-07-06T20:15:51.739892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:22.883406+0000","scrub_duration":0.0013986980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:53.747488+0000","last_change":"2023-07-06T20:15:53.747488+0000","last_active":"2023-07-06T20:15:53.747488+0000","last_peered":"2023-07-06T20:15:53.747488+0000","last_clean":"2023-07-06T20:15:53.747488+0000","last_became_active":"2023-07-06T20:13:52.980731+0000","last_became_peered":"2023-07-06T20:13:52.980731+0000","last_unstale":"2023-07-06T20:15:53.747488+0000","last_undegraded":"2023-07-06T20:15:53.747488+0000","last_fullsized":"2023-07-06T20:15:53.747488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:53.747330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:53.747330+0000","last_clean_scrub_stamp":"2023-07-06T20:15:53.747330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:45:44.935394+0000","scrub_duration":0.001721815,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"40'119","reported_seq":215,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:54.237391+0000","last_change":"2023-07-06T20:15:54.237464+0000","last_active":"2023-07-06T20:15:54.237391+0000","last_peered":"2023-07-06T20:15:54.237391+0000","last_clean":"2023-07-06T20:15:54.237391+0000","last_became_active":"2023-07-06T20:13:45.595245+0000","last_became_peered":"2023-07-06T20:13:45.595245+0000","last_unstale":"2023-07-06T20:15:54.237391+0000","last_undegraded":"2023-07-06T20:15:54.237391+0000","last_fullsized":"2023-07-06T20:15:54.237391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'119","last_scrub_stamp":"2023-07-06T20:15:54.237286+0000","last_deep_scrub":"40'119","last_deep_scrub_stamp":"2023-07-06T20:15:54.237286+0000","last_clean_scrub_stamp":"2023-07-06T20:15:54.237286+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:45:45.425350+0000","scrub_duration":0.0056032540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:16:03.156702+0000","last_change":"2023-07-06T20:16:03.156752+0000","last_active":"2023-07-06T20:16:03.156702+0000","last_peered":"2023-07-06T20:16:03.156702+0000","last_clean":"2023-07-06T20:16:03.156702+0000","last_became_active":"2023-07-06T20:13:52.980274+0000","last_became_peered":"2023-07-06T20:13:52.980274+0000","last_unstale":"2023-07-06T20:16:03.156702+0000","last_undegraded":"2023-07-06T20:16:03.156702+0000","last_fullsized":"2023-07-06T20:16:03.156702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:16:03.156611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:16:03.156611+0000","last_clean_scrub_stamp":"2023-07-06T20:16:03.156611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:47:44.205222+0000","scrub_duration":0.0022597580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:59.257859+0000","last_change":"2023-07-06T20:15:59.257859+0000","last_active":"2023-07-06T20:15:59.257859+0000","last_peered":"2023-07-06T20:15:59.257859+0000","last_clean":"2023-07-06T20:15:59.257859+0000","last_became_active":"2023-07-06T20:13:53.341683+0000","last_became_peered":"2023-07-06T20:13:53.341683+0000","last_unstale":"2023-07-06T20:15:59.257859+0000","last_undegraded":"2023-07-06T20:15:59.257859+0000","last_fullsized":"2023-07-06T20:15:59.257859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:59.257701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:59.257701+0000","last_clean_scrub_stamp":"2023-07-06T20:15:59.257701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:43:34.161530+0000","scrub_duration":0.001880409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:50.510463+0000","last_change":"2023-07-06T20:15:50.510463+0000","last_active":"2023-07-06T20:15:50.510463+0000","last_peered":"2023-07-06T20:15:50.510463+0000","last_clean":"2023-07-06T20:15:50.510463+0000","last_became_active":"2023-07-06T20:13:52.976902+0000","last_became_peered":"2023-07-06T20:13:52.976902+0000","last_unstale":"2023-07-06T20:15:50.510463+0000","last_undegraded":"2023-07-06T20:15:50.510463+0000","last_fullsized":"2023-07-06T20:15:50.510463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:50.510292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:50.510292+0000","last_clean_scrub_stamp":"2023-07-06T20:15:50.510292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:22:12.351110+0000","scrub_duration":0.002017591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-07-06T20:15:55.562310+0000","last_change":"2023-07-06T20:15:55.562310+0000","last_active":"2023-07-06T20:15:55.562310+0000","last_peered":"2023-07-06T20:15:55.562310+0000","last_clean":"2023-07-06T20:15:55.562310+0000","last_became_active":"2023-07-06T20:13:52.977480+0000","last_became_peered":"2023-07-06T20:13:52.977480+0000","last_unstale":"2023-07-06T20:15:55.562310+0000","last_undegraded":"2023-07-06T20:15:55.562310+0000","last_fullsized":"2023-07-06T20:15:55.562310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:15:55.562124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:15:55.562124+0000","last_clean_scrub_stamp":"2023-07-06T20:15:55.562124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:52:44.061683+0000","scrub_duration":0.001675129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":1060864,"data_stored":785343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92200000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.746},"min":{"1min":0.49299999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82299999999999995,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85199999999999998,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.54700000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.86799999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.73399999999999999,"15min":0.752},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83799999999999997,"5min":1.776,"15min":1.776},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.83599999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67000000000000004,"15min":0.67600000000000005},"min":{"1min":0.44800000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96099999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93600000000000005,"5min":1.089,"15min":1.089},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.51800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91000000000000003,"5min":1.02,"15min":1.02},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.59199999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89100000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.434,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.83099999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.077,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.53700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.72399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98699999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78900000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.77700000000000002,"15min":0.749},"min":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.125,"15min":1.125},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0149999999999999,"5min":1.375,"15min":1.375},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.436,"5min":0.433,"15min":0.433},"max":{"1min":0.91700000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95799999999999996,"5min":1.181,"15min":1.181},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.81200000000000006,"15min":0.81799999999999995},"min":{"1min":0.65100000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.95499999999999996,"5min":1.228,"15min":1.228},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.83699999999999997,"15min":0.85099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.196,"15min":1.196},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.316,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66300000000000003,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.52800000000000002,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.56499999999999995,"5min":1.889,"15min":1.889},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.66700000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.68799999999999994,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70199999999999996,"5min":2.226,"15min":2.226},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.84599999999999997,"15min":0.86599999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95799999999999996,"5min":1.823,"15min":1.823},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86399999999999999,"15min":0.873},"min":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0469999999999999,"5min":1.976,"15min":1.976},"last":0.94099999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85999999999999999,"15min":0.88},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96799999999999997,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.84699999999999998,"15min":0.86099999999999999},"min":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.004,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.873,"15min":0.89200000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0089999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.90100000000000002,"15min":0.92200000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.032,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.96399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":1056768,"data_stored":784324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46999999999999997,"15min":0.48199999999999998},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.502,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.379},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.42499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.495,"15min":0.505},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65600000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.55900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68100000000000005,"5min":1.008,"15min":1.008},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.57799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.871,"5min":1.369,"15min":1.369},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.56000000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.876,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.623},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.78100000000000003,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.78100000000000003,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.755},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.81399999999999995,"15min":0.82899999999999996},"min":{"1min":0.60499999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.874,"5min":1.212,"15min":1.212},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.93500000000000005,"5min":1.157,"15min":1.157},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.82399999999999995},"min":{"1min":0.57599999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93200000000000005,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.68899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.55400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.373},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.59299999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68999999999999995,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49399999999999999,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69099999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.504}]},{"osd":4,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1020000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94899999999999995,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.999}]},{"osd":5,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.90300000000000002,"5min":1.258,"15min":1.258},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1499999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.745}]},{"osd":6,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95699999999999996,"5min":1.679,"15min":1.679},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.53800000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.98399999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81599999999999995,"15min":0.82299999999999995},"min":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.089,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.90400000000000003,"15min":0.98099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60999999999999999,"5min":11.109999999999999,"15min":11.109999999999999},"last":0.501},{"interface":"front","average":{"1min":0.437,"5min":1.1990000000000001,"15min":1.3260000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61599999999999999,"5min":11.324999999999999,"15min":11.324999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.59099999999999997,"15min":0.61399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.60999999999999999,"5min":2.278,"15min":2.278},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.98599999999999999,"15min":1.0660000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81899999999999995,"5min":11.145,"15min":11.145},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.66000000000000003,"15min":0.68200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.754,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":1.0620000000000001,"15min":1.1479999999999999},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82299999999999995,"5min":11.664,"15min":11.664},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.456,"15min":1.5780000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.997,"5min":11.208,"15min":11.208},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":1.1559999999999999,"15min":1.226},"min":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.95199999999999996,"5min":7.2450000000000001,"15min":7.2450000000000001},"last":0.872}]},{"osd":5,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":1.2110000000000001,"15min":1.2909999999999999},"min":{"1min":0.56799999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.89000000000000001,"5min":10.997999999999999,"15min":10.997999999999999},"last":1.01},{"interface":"front","average":{"1min":0.752,"5min":1.5029999999999999,"15min":1.6279999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98899999999999999,"5min":11.286,"15min":11.286},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.93700000000000006},"min":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.2390000000000001,"15min":1.319},"min":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.93700000000000006,"5min":11.055,"15min":11.055},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":1.5069999999999999,"15min":1.629},"min":{"1min":0.65000000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.024,"5min":11.247999999999999,"15min":11.247999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.91800000000000004,"15min":0.94399999999999995},"min":{"1min":0.64100000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.95299999999999996,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.90700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.72499999999999998,"15min":0.71099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.88800000000000001,"15min":0.90900000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.9689999999999999,"5min":7.0010000000000003,"15min":7.0010000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.69999999999999996,"15min":0.67800000000000005},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.5},"max":{"1min":4.0330000000000004,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88200000000000001,"15min":0.88600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.016,"5min":4.7939999999999996,"15min":4.7939999999999996},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.76800000000000002},"min":{"1min":0.53000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":3.984,"5min":3.984,"15min":3.984},"last":0.625}]},{"osd":3,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.1500000000000004,"5min":5.1710000000000003,"15min":5.1710000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.93500000000000005,"15min":0.94499999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":4.0460000000000003,"5min":5.1200000000000001,"15min":5.1200000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":0.747}]},{"osd":6,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":3.8140000000000001,"5min":3.8140000000000001,"15min":3.8140000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.77000000000000002,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.9199999999999999,"5min":3.9199999999999999,"15min":3.9199999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.76100000000000001,"15min":0.74399999999999999},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.7549999999999999,"5min":3.7549999999999999,"15min":3.7549999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.80000000000000004,"15min":0.78500000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":466944,"data_stored":194975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68899999999999995,"15min":0.69499999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.628},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.629}]},{"osd":1,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66100000000000003,"15min":0.66000000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.76300000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.53700000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.57099999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.746},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.432,"5min":0.373,"15min":0.373},"max":{"1min":0.83199999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.58899999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.71299999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93799999999999994,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.68700000000000006,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:16:14.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-06T20:16:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.988+0000 7fb65ddff700 1 Processor -- start 2023-07-06T20:16:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.989+0000 7fb65ddff700 1 -- start start 2023-07-06T20:16:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb65ddff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb65815fca0 0x7fb6580d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb65ddff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb6580cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb65ddff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6580cf780 con 0x7fb6580b5c90 2023-07-06T20:16:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb65ddff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6580cd490 con 0x7fb65815fca0 2023-07-06T20:16:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb65ddff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580cd770 con 0x7fb6580b6110 2023-07-06T20:16:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb6577fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb65815fca0 0x7fb6580d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb64ffff700 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb6580b5c90 0x7fb6580d2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34066/0 (socket says 172.21.15.27:34066) 2023-07-06T20:16:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb6577fe700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb65815fca0 0x7fb6580d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53988/0 (socket says 172.21.15.27:53988) 2023-07-06T20:16:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.990+0000 7fb64ffff700 1 -- 172.21.15.27:0/3334293038 learned_addr learned my addr 172.21.15.27:0/3334293038 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-07-06T20:16:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb6577fe700 1 -- 172.21.15.27:0/3334293038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 msgr2=0x7fb6580cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:16:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb657fff700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb6580cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb656ffd700 1 -- 172.21.15.27:0/3334293038 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230340605 0 0) 0x7fb6580cf780 con 0x7fb6580b5c90 2023-07-06T20:16:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb6577fe700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb6580cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb6577fe700 1 -- 172.21.15.27:0/3334293038 >> v1:172.21.15.27:6789/0 conn(0x7fb6580b5c90 legacy=0x7fb6580d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb6577fe700 1 -- 172.21.15.27:0/3334293038 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580cda50 con 0x7fb65815fca0 2023-07-06T20:16:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb657fff700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb6580cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:16:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb6577fe700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580d1420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb648009840 tx=0x7fb648009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a00661cb896e0c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb656ffd700 1 -- 172.21.15.27:0/3334293038 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648011830 con 0x7fb65815fca0 2023-07-06T20:16:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb656ffd700 1 -- 172.21.15.27:0/3334293038 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb648011990 con 0x7fb65815fca0 2023-07-06T20:16:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.991+0000 7fb656ffd700 1 -- 172.21.15.27:0/3334293038 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648011d20 con 0x7fb65815fca0 2023-07-06T20:16:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 -- 172.21.15.27:0/3334293038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 msgr2=0x7fb6580d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580d1420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb648009840 tx=0x7fb648009b50 comp rx=0 tx=0).stop 2023-07-06T20:16:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 -- 172.21.15.27:0/3334293038 shutdown_connections 2023-07-06T20:16:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb6580cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 --2- 172.21.15.27:0/3334293038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580d1420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 -- 172.21.15.27:0/3334293038 >> 172.21.15.27:0/3334293038 conn(0x7fb65806c150 msgr2=0x7fb65806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 -- 172.21.15.27:0/3334293038 shutdown_connections 2023-07-06T20:16:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.992+0000 7fb65ddff700 1 -- 172.21.15.27:0/3334293038 wait complete. 2023-07-06T20:16:15.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.994+0000 7fb65ddff700 1 Processor -- start 2023-07-06T20:16:15.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.994+0000 7fb65ddff700 1 -- start start 2023-07-06T20:16:15.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb65ddff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6580b5c90 0x7fb6580c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb65ddff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb65814a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb65ddff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb65ddff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580ccd70 con 0x7fb65815fca0 2023-07-06T20:16:15.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb65ddff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580c7880 con 0x7fb6580b5c90 2023-07-06T20:16:15.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb65ddff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580c7ab0 con 0x7fb6580b6110 2023-07-06T20:16:15.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb657fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb657fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53996/0 (socket says 172.21.15.27:53996) 2023-07-06T20:16:15.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb657fff700 1 -- 172.21.15.27:0/1534576671 learned_addr learned my addr 172.21.15.27:0/1534576671 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:16:15.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb64ffff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb65814a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb6577fe700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6580b5c90 0x7fb6580c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb657fff700 1 -- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 msgr2=0x7fb65814a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.995+0000 7fb657fff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb65814a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb657fff700 1 -- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6580b5c90 msgr2=0x7fb6580c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb657fff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6580b5c90 0x7fb6580c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb657fff700 1 -- 172.21.15.27:0/1534576671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480094a0 con 0x7fb65815fca0 2023-07-06T20:16:15.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb64ffff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb65814a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:16:15.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb657fff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580c83e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb65000cef0 tx=0x7fb65000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:15.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000adb0 con 0x7fb65815fca0 2023-07-06T20:16:15.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650015440 con 0x7fb65815fca0 2023-07-06T20:16:15.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6580c9fb0 con 0x7fb65815fca0 2023-07-06T20:16:15.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500157f0 con 0x7fb65815fca0 2023-07-06T20:16:15.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.996+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6580ca570 con 0x7fb65815fca0 2023-07-06T20:16:15.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.997+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6580008d0 con 0x7fb65815fca0 2023-07-06T20:16:15.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.998+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb65001f440 con 0x7fb65815fca0 2023-07-06T20:16:15.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.999+0000 7fb654ff9700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63806ace0 0x7fb63806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:14.999+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb6500896b0 con 0x7fb65815fca0 2023-07-06T20:16:15.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.000+0000 7fb6577fe700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63806ace0 0x7fb63806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.000+0000 7fb6577fe700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63806ace0 0x7fb63806d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb6480090b0 tx=0x7fb64801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:15.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.004+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb650054a70 con 0x7fb65815fca0 2023-07-06T20:16:15.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.213+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fb65815f4c0 con 0x7fb65815fca0 2023-07-06T20:16:15.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.218+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb650007a00 con 0x7fb65815fca0 2023-07-06T20:16:15.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.218+0000 7fb654ff9700 1 -- 172.21.15.27:0/1534576671 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb650058090 con 0x7fb65815fca0 2023-07-06T20:16:15.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63806ace0 msgr2=0x7fb63806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63806ace0 0x7fb63806d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb6480090b0 tx=0x7fb64801c040 comp rx=0 tx=0).stop 2023-07-06T20:16:15.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 msgr2=0x7fb6580c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580c83e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb65000cef0 tx=0x7fb65000cf20 comp rx=0 tx=0).stop 2023-07-06T20:16:15.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 shutdown_connections 2023-07-06T20:16:15.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7fb63806ace0 0x7fb63806d190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65815fca0 0x7fb6580c83e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6580b5c90 0x7fb6580c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 --2- 172.21.15.27:0/1534576671 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb6580b6110 0x7fb65814a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 >> 172.21.15.27:0/1534576671 conn(0x7fb65806c150 msgr2=0x7fb6580b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:15.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 shutdown_connections 2023-07-06T20:16:15.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.221+0000 7fb65ddff700 1 -- 172.21.15.27:0/1534576671 wait complete. 2023-07-06T20:16:15.241 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T20:16:15.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:16:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.442+0000 7f1654dff700 1 Processor -- start 2023-07-06T20:16:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.442+0000 7f1654dff700 1 -- start start 2023-07-06T20:16:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.443+0000 7f1654dff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f165015fca0 0x7f16500d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.443+0000 7f1654dff700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f16500b5c90 0x7f16500d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.443+0000 7f1654dff700 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16500cf780 con 0x7f16500b6110 2023-07-06T20:16:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.443+0000 7f1654dff700 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16500cd490 con 0x7f16500b5c90 2023-07-06T20:16:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.443+0000 7f1654dff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16500cd770 con 0x7f165015fca0 2023-07-06T20:16:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.443+0000 7f164dd9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f16500b5c90 0x7f16500d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f16500b5c90 0x7f16500d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53998/0 (socket says 172.21.15.27:53998) 2023-07-06T20:16:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 -- 172.21.15.27:0/477709048 learned_addr learned my addr 172.21.15.27:0/477709048 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:16:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164e59c700 1 --2- 172.21.15.27:0/477709048 >> v2:172.21.15.153:3300/0 conn(0x7f165015fca0 0x7f16500d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 -- 172.21.15.27:0/477709048 >> v2:172.21.15.153:3300/0 conn(0x7f165015fca0 msgr2=0x7f16500d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:16:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 --2- 172.21.15.27:0/477709048 >> v2:172.21.15.153:3300/0 conn(0x7f165015fca0 0x7f16500d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 -- 172.21.15.27:0/477709048 >> v1:172.21.15.27:6789/0 conn(0x7f16500b6110 legacy=0x7f16500cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 -- 172.21.15.27:0/477709048 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16500cda50 con 0x7f16500b5c90 2023-07-06T20:16:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164e59c700 1 --2- 172.21.15.27:0/477709048 >> v2:172.21.15.153:3300/0 conn(0x7f165015fca0 0x7f16500d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:16:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164d59a700 1 -- 172.21.15.27:0/477709048 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579984833 0 0) 0x7f16500cf780 con 0x7f16500b6110 2023-07-06T20:16:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.444+0000 7f164dd9b700 1 --2- 172.21.15.27:0/477709048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16500b5c90 0x7f16500d2d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f16500d0930 tx=0x7f1644009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd6a4ffca7c7d60 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:15.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f164d59a700 1 -- 172.21.15.27:0/477709048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400b870 con 0x7f16500b5c90 2023-07-06T20:16:15.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f164d59a700 1 -- 172.21.15.27:0/477709048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f164400b9d0 con 0x7f16500b5c90 2023-07-06T20:16:15.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f164d59a700 1 -- 172.21.15.27:0/477709048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400bd80 con 0x7f16500b5c90 2023-07-06T20:16:15.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 -- 172.21.15.27:0/477709048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16500b5c90 msgr2=0x7f16500d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 --2- 172.21.15.27:0/477709048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16500b5c90 0x7f16500d2d40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f16500d0930 tx=0x7f1644009bd0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 -- 172.21.15.27:0/477709048 shutdown_connections 2023-07-06T20:16:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 --2- 172.21.15.27:0/477709048 >> v2:172.21.15.153:3300/0 conn(0x7f165015fca0 0x7f16500d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 --2- 172.21.15.27:0/477709048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16500b5c90 0x7f16500d2d40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 -- 172.21.15.27:0/477709048 >> 172.21.15.27:0/477709048 conn(0x7f165006c150 msgr2=0x7f165006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 -- 172.21.15.27:0/477709048 shutdown_connections 2023-07-06T20:16:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.445+0000 7f1654dff700 1 -- 172.21.15.27:0/477709048 wait complete. 2023-07-06T20:16:15.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.447+0000 7f1654dff700 1 Processor -- start 2023-07-06T20:16:15.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.447+0000 7f1654dff700 1 -- start start 2023-07-06T20:16:15.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f1654dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16500b5c90 0x7f16500c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f1654dff700 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16500b6110 0x7f1650155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f1654dff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 0x7f16500c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f1654dff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1650156070 con 0x7f165015fca0 2023-07-06T20:16:15.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f1654dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1650155ab0 con 0x7f16500b5c90 2023-07-06T20:16:15.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f1654dff700 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1650155d90 con 0x7f16500b6110 2023-07-06T20:16:15.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f164ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 0x7f16500c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f164ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 0x7f16500c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54006/0 (socket says 172.21.15.27:54006) 2023-07-06T20:16:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f164ed9d700 1 -- 172.21.15.27:0/3475298449 learned_addr learned my addr 172.21.15.27:0/3475298449 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-07-06T20:16:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.448+0000 7f164e59c700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16500b5c90 0x7f16500c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164dd9b700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16500b6110 0x7f1650155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164ed9d700 1 -- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16500b6110 msgr2=0x7f1650155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164ed9d700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16500b6110 0x7f1650155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164ed9d700 1 -- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16500b5c90 msgr2=0x7f16500c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164ed9d700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16500b5c90 0x7f16500c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164ed9d700 1 -- 172.21.15.27:0/3475298449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644009550 con 0x7f165015fca0 2023-07-06T20:16:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f164ed9d700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 0x7f16500c7280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1648001560 tx=0x7f164800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648003e30 con 0x7f165015fca0 2023-07-06T20:16:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1648005070 con 0x7f165015fca0 2023-07-06T20:16:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16500c9fb0 con 0x7f165015fca0 2023-07-06T20:16:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648005440 con 0x7f165015fca0 2023-07-06T20:16:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.449+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16500c9690 con 0x7f165015fca0 2023-07-06T20:16:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.451+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16500ca4d0 con 0x7f165015fca0 2023-07-06T20:16:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.451+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16480055a0 con 0x7f165015fca0 2023-07-06T20:16:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.452+0000 7f163effd700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f163006ace0 0x7f163006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.452+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1648088a30 con 0x7f165015fca0 2023-07-06T20:16:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.453+0000 7f164e59c700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f163006ace0 0x7f163006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.453+0000 7f164e59c700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f163006ace0 0x7f163006d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1640009930 tx=0x7f1640008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.457+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16500ca4d0 con 0x7f165015fca0 2023-07-06T20:16:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.707+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16500ca4d0 con 0x7f165015fca0 2023-07-06T20:16:15.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.707+0000 7f163effd700 1 -- 172.21.15.27:0/3475298449 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f16500ca4d0 con 0x7f165015fca0 2023-07-06T20:16:15.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-07-06T20:16:15.710 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4847,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2380697176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2380697176},{"type":"v1","addr":"172.21.15.153:6837","nonce":2380697176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/785263348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":785263348},{"type":"v1","addr":"172.21.15.153:6836","nonce":785263348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5136,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1149313385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1149313385},{"type":"v1","addr":"172.21.15.27:6834","nonce":1149313385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2306110977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2306110977},{"type":"v1","addr":"172.21.15.27:6835","nonce":2306110977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-07-06T20:16:15.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.709+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f163006ace0 msgr2=0x7f163006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.709+0000 7f1654dff700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f163006ace0 0x7f163006d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1640009930 tx=0x7f1640008040 comp rx=0 tx=0).stop 2023-07-06T20:16:15.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.709+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 msgr2=0x7f16500c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:15.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.709+0000 7f1654dff700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 0x7f16500c7280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1648001560 tx=0x7f164800cc00 comp rx=0 tx=0).stop 2023-07-06T20:16:15.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 shutdown_connections 2023-07-06T20:16:15.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:6832/310108246,v1:172.21.15.153:6833/310108246] conn(0x7f163006ace0 0x7f163006d190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f165015fca0 0x7f16500c7280 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16500b5c90 0x7f16500c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 --2- 172.21.15.27:0/3475298449 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f16500b6110 0x7f1650155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:15.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 >> 172.21.15.27:0/3475298449 conn(0x7f165006c150 msgr2=0x7f16500b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:15.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 shutdown_connections 2023-07-06T20:16:15.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-07-06T20:16:15.710+0000 7f1654dff700 1 -- 172.21.15.27:0/3475298449 wait complete. 2023-07-06T20:16:15.714 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2023-07-06T20:16:15.729 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-06T20:16:15.730 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-06T20:16:15.730 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T20:16:15.730 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:15.743 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T20:16:15.744 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T20:16:15.744 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:15.797 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T20:16:15.797 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T20:16:15.798 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:15.814 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T20:16:15.815 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T20:16:15.815 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:15.871 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T20:16:15.871 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-06T20:16:15.872 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-06T20:16:15.872 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:15.911 INFO:tasks.ceph.osd.0:Stopped 2023-07-06T20:16:15.911 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-06T20:16:15.912 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:15.991 INFO:tasks.ceph.osd.1:Stopped 2023-07-06T20:16:15.992 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-06T20:16:15.993 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.075 INFO:tasks.ceph.osd.2:Stopped 2023-07-06T20:16:16.075 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-06T20:16:16.076 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.126 INFO:tasks.ceph.osd.3:Stopped 2023-07-06T20:16:16.127 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-06T20:16:16.127 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.173 INFO:tasks.ceph.osd.4:Stopped 2023-07-06T20:16:16.173 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-06T20:16:16.174 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.218 INFO:tasks.ceph.osd.5:Stopped 2023-07-06T20:16:16.218 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-06T20:16:16.219 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.267 INFO:tasks.ceph.osd.6:Stopped 2023-07-06T20:16:16.268 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-06T20:16:16.268 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.307 INFO:tasks.ceph.osd.7:Stopped 2023-07-06T20:16:16.308 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-06T20:16:16.309 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-06T20:16:16.309 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.366 INFO:tasks.ceph.mgr.y:Stopped 2023-07-06T20:16:16.367 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-06T20:16:16.367 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.430 INFO:tasks.ceph.mgr.x:Stopped 2023-07-06T20:16:16.430 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-06T20:16:16.430 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-06T20:16:16.431 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.449 INFO:tasks.ceph.mon.a:Stopped 2023-07-06T20:16:16.450 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-06T20:16:16.450 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.467 INFO:tasks.ceph.mon.b:Stopped 2023-07-06T20:16:16.467 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-06T20:16:16.468 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T20:16:16.523 INFO:tasks.ceph.mon.c:Stopped 2023-07-06T20:16:16.524 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-06T20:16:16.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-07-06T20:16:16.607 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:16:16.608 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-06T20:16:16.908 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:16:16.908 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-06T20:16:17.119 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:16:17.120 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-06T20:16:17.320 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:16:17.321 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-06T20:16:17.508 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:16:17.509 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-06T20:16:18.052 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:16:18.053 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-06T20:16:18.253 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:16:18.253 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-06T20:16:18.462 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:16:18.463 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-06T20:16:18.651 INFO:tasks.ceph:Archiving mon data... 2023-07-06T20:16:18.653 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/data/mon.a.tgz 2023-07-06T20:16:18.653 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-07-06T20:16:18.704 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.IT3UeU3XHb 2023-07-06T20:16:18.705 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IT3UeU3XHb 2023-07-06T20:16:18.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.IT3UeU3XHb 2023-07-06T20:16:18.972 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.IT3UeU3XHb is 291KB 2023-07-06T20:16:19.069 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.IT3UeU3XHb 2023-07-06T20:16:19.115 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/data/mon.b.tgz 2023-07-06T20:16:19.115 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-07-06T20:16:19.161 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.nXpNMzerMI 2023-07-06T20:16:19.162 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nXpNMzerMI 2023-07-06T20:16:19.351 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.nXpNMzerMI 2023-07-06T20:16:19.466 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.nXpNMzerMI is 389KB 2023-07-06T20:16:19.562 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.nXpNMzerMI 2023-07-06T20:16:19.607 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/data/mon.c.tgz 2023-07-06T20:16:19.608 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-07-06T20:16:19.692 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.HSkhmDM49m 2023-07-06T20:16:19.693 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HSkhmDM49m 2023-07-06T20:16:19.876 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.HSkhmDM49m 2023-07-06T20:16:19.990 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.HSkhmDM49m is 389KB 2023-07-06T20:16:20.087 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.HSkhmDM49m 2023-07-06T20:16:20.131 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-06T20:16:20.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T20:16:20.137 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T20:16:20.239 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-06T20:16:20.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T20:16:20.244 DEBUG:teuthology.orchestra.run.smithi153:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T20:16:20.323 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T20:16:20.325 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T20:16:20.341 INFO:tasks.ceph:Archiving crash dumps... 2023-07-06T20:16:20.343 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/remote/smithi027/crash 2023-07-06T20:16:20.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T20:16:20.415 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/remote/smithi153/crash 2023-07-06T20:16:20.416 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T20:16:20.483 INFO:tasks.ceph:Shutting down logrotate 2023-07-06T20:16:20.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:16:20.487 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T20:16:20.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T20:16:20.654 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T20:16:20.722 INFO:tasks.ceph:Compressing logs... 2023-07-06T20:16:20.723 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T20:16:20.727 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T20:16:20.792 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119594.log 2023-07-06T20:16:20.793 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124185.log 2023-07-06T20:16:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125988.log 88.2% 2023-07-06T20:16:20.794 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119594.log.gz 2023-07-06T20:16:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130574.log 2023-07-06T20:16:20.796 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124185.log.gz 2023-07-06T20:16:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-06T20:16:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112085.log 2023-07-06T20:16:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125988.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125988.log.gz 2023-07-06T20:16:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131185.log 2023-07-06T20:16:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130574.log.gz 2023-07-06T20:16:20.798 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126881.log 2023-07-06T20:16:20.799 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.131185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129377.log 2023-07-06T20:16:20.800 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131185.log.gz 2023-07-06T20:16:20.800 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126881.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126881.log.gz 2023-07-06T20:16:20.801 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112234.log 2023-07-06T20:16:20.801 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112284.log 2023-07-06T20:16:20.801 INFO:teuthology.orchestra.run.smithi153.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112085.log.gz 2023-07-06T20:16:20.801 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112234.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112234.log.gz 2023-07-06T20:16:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T20:16:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112284.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112284.log.gz 2023-07-06T20:16:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112332.log 2023-07-06T20:16:20.804 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.119475.log 2023-07-06T20:16:20.804 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112332.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112332.log.gz 2023-07-06T20:16:20.804 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-06T20:16:20.804 INFO:teuthology.orchestra.run.smithi153.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T20:16:20.805 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.3.119475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.119475.log.gz 2023-07-06T20:16:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131136.log 2023-07-06T20:16:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119867.log 2023-07-06T20:16:20.806 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129377.log.gz 2023-07-06T20:16:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.131136.log: gzip -5 --verbose 88.3%-- /var/log/ceph/ceph-client.admin.124430.log -- replaced with /var/log/ceph/ceph-client.admin.131136.log.gz 2023-07-06T20:16:20.806 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-07-06T20:16:20.807 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127605.log 2023-07-06T20:16:20.807 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119867.log.gz 2023-07-06T20:16:20.807 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124430.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.124430.log.gz 2023-07-06T20:16:20.807 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125792.log 2023-07-06T20:16:20.807 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127605.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127605.log.gz 2023-07-06T20:16:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118618.log 2023-07-06T20:16:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125792.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125792.log.gz 2023-07-06T20:16:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130892.log 2023-07-06T20:16:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-06T20:16:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112133.log 2023-07-06T20:16:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-06T20:16:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112133.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112133.log.gz 2023-07-06T20:16:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112381.log 2023-07-06T20:16:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112036.log 2023-07-06T20:16:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112381.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112381.log.gz 2023-07-06T20:16:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-06T20:16:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118618.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118618.log.gz 2023-07-06T20:16:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127458.log 2023-07-06T20:16:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130892.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130892.log.gz 2023-07-06T20:16:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119980.log 2023-07-06T20:16:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127458.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127458.log.gz 2023-07-06T20:16:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131039.log 2023-07-06T20:16:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119980.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119980.log.gz 2023-07-06T20:16:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126487.log 2023-07-06T20:16:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.131039.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131039.log.gz 2023-07-06T20:16:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121151.log 2023-07-06T20:16:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126487.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126487.log.gz 2023-07-06T20:16:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123112.log 2023-07-06T20:16:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.121151.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121151.log.gz 2023-07-06T20:16:20.816 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129591.log 2023-07-06T20:16:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123112.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123112.log.gz 2023-07-06T20:16:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117755.log 2023-07-06T20:16:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129591.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129591.log.gz 2023-07-06T20:16:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128808.log 2023-07-06T20:16:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.117755.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117755.log.gz 2023-07-06T20:16:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130794.log 2023-07-06T20:16:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128808.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128808.log.gz 2023-07-06T20:16:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119979.log 2023-07-06T20:16:20.820 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130794.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130794.log.gz 2023-07-06T20:16:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125351.log 2023-07-06T20:16:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119979.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119979.log.gz 2023-07-06T20:16:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124796.log 2023-07-06T20:16:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125351.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125351.log.gz 2023-07-06T20:16:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126734.log 2023-07-06T20:16:20.822 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112036.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-06T20:16:20.822 INFO:teuthology.orchestra.run.smithi153.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112036.log.gz 2023-07-06T20:16:20.822 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112185.log 2023-07-06T20:16:20.822 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-06T20:16:20.823 INFO:teuthology.orchestra.run.smithi153.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-06T20:16:20.823 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.admin.112185.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112185.log.gz 2023-07-06T20:16:20.823 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T20:16:20.823 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-06T20:16:20.823 INFO:teuthology.orchestra.run.smithi153.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-06T20:16:20.823 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.120325.log 2023-07-06T20:16:20.824 INFO:teuthology.orchestra.run.smithi153.stderr: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-06T20:16:20.825 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.118796.log 2023-07-06T20:16:20.825 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.1.120325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.120325.log.gz 2023-07-06T20:16:20.825 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-06T20:16:20.825 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-client.1.118796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.118796.log.gz 2023-07-06T20:16:20.827 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124796.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124796.log.gz 2023-07-06T20:16:20.827 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118775.log 2023-07-06T20:16:20.827 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126734.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126734.log.gz 2023-07-06T20:16:20.828 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119265.log 2023-07-06T20:16:20.828 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118775.log: gzip -5 87.5%--verbose -- replaced with /var/log/ceph/ceph-client.admin.118775.log.gz 2023-07-06T20:16:20.828 INFO:teuthology.orchestra.run.smithi027.stderr:-- /var/log/ceph/ceph-client.admin.126252.log 2023-07-06T20:16:20.828 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119265.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.119265.log.gz 2023-07-06T20:16:20.828 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128417.log 2023-07-06T20:16:20.829 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126252.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126252.log.gz 2023-07-06T20:16:20.829 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123847.log 2023-07-06T20:16:20.829 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128417.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128417.log.gz 2023-07-06T20:16:20.829 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-06T20:16:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123847.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123847.log.gz 2023-07-06T20:16:20.830 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123308.log 2023-07-06T20:16:20.831 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120539.log 2023-07-06T20:16:20.831 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123308.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123308.log.gz 2023-07-06T20:16:20.831 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121053.log 2023-07-06T20:16:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120539.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120539.log.gz 2023-07-06T20:16:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121418.log 2023-07-06T20:16:20.832 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.121053.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121053.log.gz 2023-07-06T20:16:20.833 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-07-06T20:16:20.833 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.121418.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121418.log.gz 2023-07-06T20:16:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125449.log 2023-07-06T20:16:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120588.log 2023-07-06T20:16:20.835 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125449.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125449.log.gz 2023-07-06T20:16:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126086.log 2023-07-06T20:16:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120588.log.gz 2023-07-06T20:16:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125302.log 2023-07-06T20:16:20.838 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126086.log: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-06T20:16:20.839 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126086.log.gz 2023-07-06T20:16:20.839 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118467.log 2023-07-06T20:16:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128319.log 2023-07-06T20:16:20.841 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125302.log.gz 2023-07-06T20:16:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118467.log.gz 2023-07-06T20:16:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130647.log 2023-07-06T20:16:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128319.log: gzip -5 --verbose 87.5%-- -- replaced with /var/log/ceph/ceph-client.admin.128319.log.gz/var/log/ceph/ceph-client.admin.120784.log 2023-07-06T20:16:20.844 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-07-06T20:16:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130647.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130647.log.gz 2023-07-06T20:16:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125743.log 2023-07-06T20:16:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120784.log: 88.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.120784.log.gz--verbose 2023-07-06T20:16:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:-- /var/log/ceph/ceph-client.admin.124911.log 2023-07-06T20:16:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125743.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125743.log.gz 2023-07-06T20:16:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126783.log 2023-07-06T20:16:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124911.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124911.log.gz 2023-07-06T20:16:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128759.log 2023-07-06T20:16:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126783.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126783.log.gz 2023-07-06T20:16:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127287.log 2023-07-06T20:16:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130427.log 2023-07-06T20:16:20.850 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128759.log.gz 2023-07-06T20:16:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127287.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127287.log.gz-5 2023-07-06T20:16:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:--verbose -- /var/log/ceph/ceph-client.admin.123798.log 2023-07-06T20:16:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130427.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130427.log.gz 2023-07-06T20:16:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124043.log 2023-07-06T20:16:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123798.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123798.log.gz 2023-07-06T20:16:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118418.log 2023-07-06T20:16:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124043.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124043.log.gz 2023-07-06T20:16:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123504.log 2023-07-06T20:16:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118418.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118418.log.gz 2023-07-06T20:16:20.855 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127727.log 2023-07-06T20:16:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123504.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123504.log.gz 2023-07-06T20:16:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129739.log 2023-07-06T20:16:20.857 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/ceph-osd.5.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-06T20:16:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127727.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127727.log.gz 2023-07-06T20:16:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125156.log 2023-07-06T20:16:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129739.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129739.log.gz 2023-07-06T20:16:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T20:16:20.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125156.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125156.log.gz 2023-07-06T20:16:20.859 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125890.log 2023-07-06T20:16:20.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121934.log 2023-07-06T20:16:20.861 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T20:16:20.861 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125890.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125890.log.gz 2023-07-06T20:16:20.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129271.log 2023-07-06T20:16:20.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.121934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121934.log.gz 2023-07-06T20:16:20.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129173.log 2023-07-06T20:16:20.864 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124592.log 2023-07-06T20:16:20.864 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129271.log.gz 2023-07-06T20:16:20.865 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129173.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129173.log.gz 2023-07-06T20:16:20.865 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130990.log 2023-07-06T20:16:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124592.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124592.log.gz 2023-07-06T20:16:20.866 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117936.log 2023-07-06T20:16:20.867 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130990.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130990.log.gz 2023-07-06T20:16:20.867 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128107.log 2023-07-06T20:16:20.868 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.117936.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117936.log.gz 2023-07-06T20:16:20.869 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129075.log 2023-07-06T20:16:20.869 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128107.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128107.log.gz 2023-07-06T20:16:20.870 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122189.log 2023-07-06T20:16:20.871 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129075.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129075.log.gz 2023-07-06T20:16:20.871 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123553.log 2023-07-06T20:16:20.871 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.122189.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122189.log.gz 2023-07-06T20:16:20.872 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119118.log 2023-07-06T20:16:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123553.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.123553.log.gz 2023-07-06T20:16:20.873 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121249.log 2023-07-06T20:16:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119118.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119118.log.gz 2023-07-06T20:16:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124747.log 2023-07-06T20:16:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.121249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119020.log 2023-07-06T20:16:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124747.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.121249.log.gz 2023-07-06T20:16:20.876 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124747.log.gz 2023-07-06T20:16:20.877 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125841.log 2023-07-06T20:16:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130696.log 2023-07-06T20:16:20.878 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119020.log.gz 2023-07-06T20:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125841.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125841.log.gz 2023-07-06T20:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130035.log 2023-07-06T20:16:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130696.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130696.log.gz 2023-07-06T20:16:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127138.log 2023-07-06T20:16:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130035.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130035.log.gz 2023-07-06T20:16:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120882.log 2023-07-06T20:16:20.882 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127138.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127138.log.gz 2023-07-06T20:16:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126636.log 2023-07-06T20:16:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120882.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120882.log.gz 2023-07-06T20:16:20.884 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120637.log 2023-07-06T20:16:20.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126636.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126636.log.gz 2023-07-06T20:16:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119167.log 2023-07-06T20:16:20.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120637.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120637.log.gz 2023-07-06T20:16:20.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-06T20:16:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119167.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119167.log.gz 2023-07-06T20:16:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.104407.log 2023-07-06T20:16:20.888 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126203.log 2023-07-06T20:16:20.889 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.tmp-client.admin.104407.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.104407.log.gz 2023-07-06T20:16:20.889 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126538.log 2023-07-06T20:16:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126203.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126203.log.gz 2023-07-06T20:16:20.891 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129124.log 2023-07-06T20:16:20.891 INFO:teuthology.orchestra.run.smithi153.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-06T20:16:20.892 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126538.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126538.log.gz 2023-07-06T20:16:20.892 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128466.log 2023-07-06T20:16:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129124.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.129124.log.gz 2023-07-06T20:16:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120234.log 2023-07-06T20:16:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128466.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128466.log.gz 2023-07-06T20:16:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-06T20:16:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120234.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120234.log.gz 2023-07-06T20:16:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126153.log 2023-07-06T20:16:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126587.log 2023-07-06T20:16:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126153.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.126153.log.gz 2023-07-06T20:16:20.899 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130745.log 2023-07-06T20:16:20.900 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118320.log 2023-07-06T20:16:20.900 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126587.log.gz 2023-07-06T20:16:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130745.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.130745.log.gz 2023-07-06T20:16:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119216.log 2023-07-06T20:16:20.903 INFO:teuthology.orchestra.run.smithi153.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-06T20:16:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118971.log 2023-07-06T20:16:20.905 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118320.log.gz 2023-07-06T20:16:20.905 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119216.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119216.log.gz 2023-07-06T20:16:20.906 INFO:teuthology.orchestra.run.smithi153.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-06T20:16:20.908 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127507.log 2023-07-06T20:16:20.908 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119593.log 2023-07-06T20:16:20.908 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118971.log.gz 2023-07-06T20:16:20.908 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127507.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127507.log.gz 2023-07-06T20:16:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119591.log 2023-07-06T20:16:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129222.log 2023-07-06T20:16:20.910 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119593.log.gz 2023-07-06T20:16:20.910 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119591.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119591.log.gz 2023-07-06T20:16:20.910 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118516.log 2023-07-06T20:16:20.911 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125547.log 2023-07-06T20:16:20.912 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129222.log.gz 2023-07-06T20:16:20.913 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118516.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118516.log.gz 2023-07-06T20:16:20.913 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120931.log 2023-07-06T20:16:20.914 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125547.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125547.log.gz 2023-07-06T20:16:20.915 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119878.log 2023-07-06T20:16:20.916 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120931.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120931.log.gz 2023-07-06T20:16:20.916 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120686.log 2023-07-06T20:16:20.922 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126351.log 2023-07-06T20:16:20.923 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119878.log.gz 2023-07-06T20:16:20.923 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120686.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120686.log.gz 2023-07-06T20:16:20.923 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120441.log 2023-07-06T20:16:20.923 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123749.log 2023-07-06T20:16:20.924 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126351.log.gz 2023-07-06T20:16:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120441.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120441.log.gz 2023-07-06T20:16:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125400.log 2023-07-06T20:16:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125253.log 2023-07-06T20:16:20.925 INFO:teuthology.orchestra.run.smithi027.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.123749.log.gz 2023-07-06T20:16:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125400.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125400.log.gz 2023-07-06T20:16:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124234.log 2023-07-06T20:16:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125253.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125253.log.gz 2023-07-06T20:16:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123060.log 2023-07-06T20:16:20.927 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119069.log 2023-07-06T20:16:20.927 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124234.log.gz 2023-07-06T20:16:20.928 INFO:teuthology.orchestra.run.smithi027.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-06T20:16:20.928 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123060.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123060.log.gz 2023-07-06T20:16:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118724.log 2023-07-06T20:16:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125694.log 2023-07-06T20:16:20.930 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119069.log.gz 2023-07-06T20:16:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118724.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118724.log.gz 2023-07-06T20:16:20.932 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118567.log 2023-07-06T20:16:20.932 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125694.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125694.log.gz 2023-07-06T20:16:20.932 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127336.log 2023-07-06T20:16:20.933 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118567.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118567.log.gz 2023-07-06T20:16:20.933 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130280.log 2023-07-06T20:16:20.935 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127336.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127336.log.gz 2023-07-06T20:16:20.935 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130941.log 2023-07-06T20:16:20.935 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130280.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130280.log.gz 2023-07-06T20:16:20.936 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126438.log 2023-07-06T20:16:20.937 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130941.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130941.log.gz 2023-07-06T20:16:20.937 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118675.log 2023-07-06T20:16:20.938 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126438.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126438.log.gz 2023-07-06T20:16:20.938 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126685.log 2023-07-06T20:16:20.939 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118675.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118675.log.gz 2023-07-06T20:16:20.940 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128515.log 2023-07-06T20:16:20.940 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126685.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.126685.log.gz 2023-07-06T20:16:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120490.log 2023-07-06T20:16:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128515.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128515.log.gz 2023-07-06T20:16:20.942 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130182.log 2023-07-06T20:16:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120490.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120490.log.gz 2023-07-06T20:16:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124527.log 2023-07-06T20:16:20.944 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130182.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.130182.log.gz 2023-07-06T20:16:20.945 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125645.log 2023-07-06T20:16:20.945 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124527.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124527.log.gz 2023-07-06T20:16:20.946 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123210.log 2023-07-06T20:16:20.947 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125645.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125645.log.gz 2023-07-06T20:16:20.947 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122907.log 2023-07-06T20:16:20.948 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123210.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123210.log.gz 2023-07-06T20:16:20.948 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118172.log 2023-07-06T20:16:20.949 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.122907.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122907.log.gz 2023-07-06T20:16:20.950 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119985.log 2023-07-06T20:16:20.950 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118172.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118172.log.gz 2023-07-06T20:16:20.951 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129641.log 2023-07-06T20:16:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119985.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119985.log.gz 2023-07-06T20:16:20.952 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128368.log 2023-07-06T20:16:20.953 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129641.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129641.log.gz 2023-07-06T20:16:20.953 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130084.log 2023-07-06T20:16:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128368.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128368.log.gz 2023-07-06T20:16:20.954 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119575.log 2023-07-06T20:16:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130084.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130084.log.gz 2023-07-06T20:16:20.955 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127238.log 2023-07-06T20:16:20.956 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119575.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119575.log.gz 2023-07-06T20:16:20.957 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130231.log 2023-07-06T20:16:20.958 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127238.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127238.log.gz 2023-07-06T20:16:20.958 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119981.log 2023-07-06T20:16:20.959 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130231.log: 88.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.130231.log.gz--verbose 2023-07-06T20:16:20.959 INFO:teuthology.orchestra.run.smithi027.stderr:-- /var/log/ceph/ceph-client.admin.124332.log 2023-07-06T20:16:20.960 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119981.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119981.log.gz 2023-07-06T20:16:20.960 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126037.log 2023-07-06T20:16:20.961 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124332.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124332.log.gz 2023-07-06T20:16:20.962 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117843.log 2023-07-06T20:16:20.963 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123406.log 2023-07-06T20:16:20.963 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126037.log.gz 2023-07-06T20:16:20.964 INFO:teuthology.orchestra.run.smithi153.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-06T20:16:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.117843.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117843.log.gz 2023-07-06T20:16:20.964 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123602.log 2023-07-06T20:16:20.965 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119590.log 88.1% 2023-07-06T20:16:20.965 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123406.log.gz 2023-07-06T20:16:20.966 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123602.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123602.log.gz 2023-07-06T20:16:20.967 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124698.log 2023-07-06T20:16:20.967 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129026.log 2023-07-06T20:16:20.968 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119590.log.gz 2023-07-06T20:16:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124698.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124698.log.gz 2023-07-06T20:16:20.969 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119592.log 2023-07-06T20:16:20.970 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129026.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129026.log.gz 2023-07-06T20:16:20.970 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129788.log 2023-07-06T20:16:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119592.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119592.log.gz 2023-07-06T20:16:20.971 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124381.log 2023-07-06T20:16:20.972 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120980.log 2023-07-06T20:16:20.973 INFO:teuthology.orchestra.run.smithi027.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.129788.log.gz 2023-07-06T20:16:20.974 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124381.log.gz 2023-07-06T20:16:20.974 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119986.log 2023-07-06T20:16:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120980.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.120980.log.gz 2023-07-06T20:16:20.975 INFO:teuthology.orchestra.run.smithi027.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.119314.log 2023-07-06T20:16:20.976 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119986.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119986.log.gz 2023-07-06T20:16:20.976 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129426.log 2023-07-06T20:16:20.977 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119314.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119314.log.gz 2023-07-06T20:16:20.977 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130133.log 2023-07-06T20:16:20.978 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129426.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129426.log.gz 2023-07-06T20:16:20.979 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118873.log 2023-07-06T20:16:20.979 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130133.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130133.log.gz 2023-07-06T20:16:20.980 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123994.log 2023-07-06T20:16:20.980 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118873.log.gz 2023-07-06T20:16:20.981 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126979.log 2023-07-06T20:16:20.982 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123994.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123994.log.gz 2023-07-06T20:16:20.982 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129524.log 2023-07-06T20:16:20.983 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126979.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126979.log.gz 2023-07-06T20:16:20.983 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-06T20:16:20.984 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129524.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129524.log.gz 2023-07-06T20:16:20.985 INFO:teuthology.orchestra.run.smithi153.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-06T20:16:20.985 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-06T20:16:20.986 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125596.log 2023-07-06T20:16:20.987 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123006.log 2023-07-06T20:16:20.989 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123161.log 2023-07-06T20:16:20.989 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125596.log.gz 2023-07-06T20:16:20.990 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123006.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123006.log.gz 2023-07-06T20:16:20.991 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117985.log 2023-07-06T20:16:20.994 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123161.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123161.log.gz 2023-07-06T20:16:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-06T20:16:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.117985.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117985.log.gz 2023-07-06T20:16:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128710.log 2023-07-06T20:16:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118922.log 2023-07-06T20:16:20.997 INFO:teuthology.orchestra.run.smithi153.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-06T20:16:20.997 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128710.log: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.128710.log.gz--verbose 2023-07-06T20:16:20.998 INFO:teuthology.orchestra.run.smithi027.stderr:-- /var/log/ceph/ceph-client.admin.124649.log 2023-07-06T20:16:20.998 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118922.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118922.log.gz 2023-07-06T20:16:20.999 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.122662.log 2023-07-06T20:16:21.000 INFO:teuthology.orchestra.run.smithi153.stderr: 2023-07-06T20:16:21.000 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m0.230s 2023-07-06T20:16:21.000 INFO:teuthology.orchestra.run.smithi153.stderr:user 0m0.794s 2023-07-06T20:16:21.001 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m0.143s 2023-07-06T20:16:21.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124649.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124649.log.gz 2023-07-06T20:16:21.002 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124283.log 2023-07-06T20:16:21.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.2.122662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.122662.log.gz 2023-07-06T20:16:21.002 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127987.log 2023-07-06T20:16:21.003 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124283.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124283.log.gz 2023-07-06T20:16:21.004 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112361.log 2023-07-06T20:16:21.004 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.127987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.127987.log.gz 2023-07-06T20:16:21.005 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130476.log 2023-07-06T20:16:21.006 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.112361.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112361.log.gz 2023-07-06T20:16:21.007 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T20:16:21.008 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130476.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130476.log.gz 2023-07-06T20:16:21.009 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118824.log 2023-07-06T20:16:21.011 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.log: gzip -5 84.6%--verbose -- replaced with /var/log/ceph/ceph.log.gz-- 2023-07-06T20:16:21.011 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120230.log 2023-07-06T20:16:21.012 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118824.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118824.log.gz 2023-07-06T20:16:21.012 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123259.log 2023-07-06T20:16:21.013 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120230.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120230.log.gz 2023-07-06T20:16:21.014 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120274.log 2023-07-06T20:16:21.015 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123259.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123259.log.gz 2023-07-06T20:16:21.016 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121102.log 2023-07-06T20:16:21.017 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120274.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.120274.log.gz 2023-07-06T20:16:21.017 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128661.log 2023-07-06T20:16:21.018 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.121102.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121102.log.gz 2023-07-06T20:16:21.019 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118221.log 2023-07-06T20:16:21.020 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128661.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128661.log.gz 2023-07-06T20:16:21.020 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119581.log 2023-07-06T20:16:21.022 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118221.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118221.log.gz 2023-07-06T20:16:21.023 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118084.log 2023-07-06T20:16:21.024 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119581.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119581.log.gz 2023-07-06T20:16:21.024 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127081.log 2023-07-06T20:16:21.025 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118084.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118084.log.gz 2023-07-06T20:16:21.026 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128977.log 2023-07-06T20:16:21.027 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127081.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127081.log.gz 2023-07-06T20:16:21.028 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128857.log 2023-07-06T20:16:21.028 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128977.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128977.log.gz 2023-07-06T20:16:21.029 INFO:teuthology.orchestra.run.smithi027.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-06T20:16:21.029 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120735.log 2023-07-06T20:16:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128857.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128857.log.gz 2023-07-06T20:16:21.031 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119576.log 2023-07-06T20:16:21.031 INFO:teuthology.orchestra.run.smithi027.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-06T20:16:21.032 INFO:teuthology.orchestra.run.smithi027.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-06T20:16:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120735.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.120735.log.gz 2023-07-06T20:16:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125498.log 2023-07-06T20:16:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127556.log 2023-07-06T20:16:21.034 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119576.log.gz 2023-07-06T20:16:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125498.log.gz 2023-07-06T20:16:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127030.log 2023-07-06T20:16:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127556.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127556.log.gz 2023-07-06T20:16:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130525.log 2023-07-06T20:16:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127030.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127030.log.gz 2023-07-06T20:16:21.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119984.log 2023-07-06T20:16:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130525.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130525.log.gz 2023-07-06T20:16:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124960.log 2023-07-06T20:16:21.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119984.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119984.log.gz 2023-07-06T20:16:21.040 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125107.log 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.124960.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124960.log.gz 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130329.log 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125107.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125107.log.gz 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125939.log 2023-07-06T20:16:21.043 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130329.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130329.log.gz 2023-07-06T20:16:21.043 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123357.log 2023-07-06T20:16:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125939.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125939.log.gz 2023-07-06T20:16:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123896.log 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123357.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123357.log.gz 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118369.log 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123896.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123896.log.gz 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121200.log 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118369.log: 95.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118369.log.gz 2023-07-06T20:16:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112233.log 2023-07-06T20:16:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.121200.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121200.log.gz 2023-07-06T20:16:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126930.log 2023-07-06T20:16:21.050 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.112233.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112233.log.gz 2023-07-06T20:16:21.050 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123945.log 2023-07-06T20:16:21.051 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126930.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126930.log.gz 2023-07-06T20:16:21.051 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128906.log 2023-07-06T20:16:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123945.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123945.log.gz 2023-07-06T20:16:21.052 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125009.log 2023-07-06T20:16:21.053 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128906.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128906.log.gz 2023-07-06T20:16:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120392.log 2023-07-06T20:16:21.054 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125009.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125009.log.gz 2023-07-06T20:16:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126832.log 2023-07-06T20:16:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120392.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120392.log.gz 2023-07-06T20:16:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120833.log 2023-07-06T20:16:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.126832.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126832.log.gz 2023-07-06T20:16:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130843.log 2023-07-06T20:16:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120833.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.120833.log.gz--verbose 2023-07-06T20:16:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:-- /var/log/ceph/ceph-client.admin.119363.log 2023-07-06T20:16:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130843.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130843.log.gz 2023-07-06T20:16:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130378.log 2023-07-06T20:16:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.119363.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119363.log.gz 2023-07-06T20:16:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128564.log 2023-07-06T20:16:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.130378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130378.log.gz 2023-07-06T20:16:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111890.log 2023-07-06T20:16:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.128564.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128564.log.gz 2023-07-06T20:16:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-07-06T20:16:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.111890.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111890.log.gz 2023-07-06T20:16:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123455.log 2023-07-06T20:16:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129690.log 2023-07-06T20:16:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123455.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123455.log.gz 2023-07-06T20:16:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122956.log 2023-07-06T20:16:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129690.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129690.log.gz 2023-07-06T20:16:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125058.log 2023-07-06T20:16:21.069 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.122956.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122956.log.gz 2023-07-06T20:16:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127409.log 2023-07-06T20:16:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.125058.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125058.log.gz 2023-07-06T20:16:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127654.log 2023-07-06T20:16:21.072 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127409.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127409.log.gz 2023-07-06T20:16:21.072 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127187.log 2023-07-06T20:16:21.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118271.log 2023-07-06T20:16:21.074 INFO:teuthology.orchestra.run.smithi027.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.127654.log.gz 2023-07-06T20:16:21.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.127187.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127187.log.gz 2023-07-06T20:16:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131089.log 2023-07-06T20:16:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.118271.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118271.log.gz 2023-07-06T20:16:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123700.log 2023-07-06T20:16:21.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.131089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129320.log 2023-07-06T20:16:21.077 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131089.log.gz 2023-07-06T20:16:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123700.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123700.log.gz 2023-07-06T20:16:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123651.log 2023-07-06T20:16:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129320.log.gz 2023-07-06T20:16:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120273.log 2023-07-06T20:16:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.123651.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.123651.log.gz--verbose 2023-07-06T20:16:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:-- /var/log/ceph/ceph-client.admin.129475.log 2023-07-06T20:16:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.120273.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120273.log.gz 2023-07-06T20:16:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.129475.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129475.log.gz 2023-07-06T20:16:21.177 INFO:teuthology.orchestra.run.smithi027.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-06T20:16:21.197 INFO:teuthology.orchestra.run.smithi027.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-06T20:16:21.201 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-07-06T20:16:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m0.435s 2023-07-06T20:16:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m0.823s 2023-07-06T20:16:21.201 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.635s 2023-07-06T20:16:21.202 INFO:tasks.ceph:Archiving logs... 2023-07-06T20:16:21.203 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/remote/smithi027/log 2023-07-06T20:16:21.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T20:16:21.502 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/remote/smithi153/log 2023-07-06T20:16:21.503 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T20:16:21.663 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-06T20:16:21.685 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-06T20:16:21.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T20:16:21.690 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T20:16:21.823 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-06T20:16:21.834 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-06T20:16:21.835 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:16:21.839 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:16:21.879 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2023-07-06T20:16:21.884 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-07-06T20:16:21.887 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:16:21.887 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-07-06T20:16:21.887 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +1556us[+1556us] +/- 77ms 2023-07-06T20:16:21.888 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -994us[ -996us] +/- 54ms 2023-07-06T20:16:21.888 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -166us[ -166us] +/- 71ms 2023-07-06T20:16:21.888 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:16:21.891 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:16:21.892 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-07-06T20:16:21.892 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +1613us[+1624us] +/- 77ms 2023-07-06T20:16:21.892 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -955us[ -945us] +/- 54ms 2023-07-06T20:16:21.892 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -133us[ -123us] +/- 71ms 2023-07-06T20:16:21.893 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:16:21.894 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-06T20:16:21.904 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-06T20:16:21.904 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-06T20:16:21.915 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T20:16:22.101 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T20:16:22.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T20:16:22.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:16:22.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T20:16:22.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:16:22.415 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-06T20:16:22.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-06T20:16:22.439 INFO:teuthology.task.internal:Duration was 1327.894639 seconds 2023-07-06T20:16:22.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-06T20:16:22.454 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-06T20:16:22.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T20:16:22.459 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T20:16:22.575 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T20:16:22.576 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T20:16:22.943 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-06T20:16:22.944 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2023-07-06T20:16:22.944 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T20:16:23.009 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2023-07-06T20:16:23.010 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T20:16:23.071 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-06T20:16:23.072 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T20:16:23.076 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T20:16:23.158 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-06T20:16:23.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T20:16:23.163 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T20:16:23.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-06T20:16:23.292 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-06T20:16:23.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T20:16:23.364 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T20:16:23.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-06T20:16:23.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:16:23.449 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:16:23.515 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2023-07-06T20:16:23.517 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2023-07-06T20:16:23.551 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:16:23.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:16:23.596 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:16:23.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:16:23.641 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-06T20:16:23.654 INFO:teuthology.task.internal:Transferring archived files... 2023-07-06T20:16:23.655 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/remote/smithi027 2023-07-06T20:16:23.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T20:16:23.734 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147/remote/smithi153 2023-07-06T20:16:23.734 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T20:16:23.810 INFO:teuthology.task.internal:Removing archive directory... 2023-07-06T20:16:23.810 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T20:16:23.815 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T20:16:23.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-06T20:16:23.879 INFO:teuthology.task.internal:Not uploading archives. 2023-07-06T20:16:23.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-06T20:16:23.896 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-06T20:16:23.897 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T20:16:23.902 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T20:16:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 20:16 /home/ubuntu/cephtest 2023-07-06T20:16:23.948 INFO:teuthology.orchestra.run.smithi153.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 20:16 /home/ubuntu/cephtest 2023-07-06T20:16:23.952 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-06T20:16:23.962 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-06T20:16:23.987 INFO:teuthology.nuke:Checking targets against current locks 2023-07-06T20:16:24.017 DEBUG:teuthology.nuke:shortname: smithi027 2023-07-06T20:16:24.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T20:16:24.043 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:36.815553', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvVqYBqk6RwHd+gGqqcWtRbd74UJHsWYb4z5uVCfrZeMRhCv2tuwoMsQ044WHvlZMgS/Qazj6ja8+mE7864w35fl/CNZkDN3P4U7bwP8AAD5ZLAfagzgMFWLN+x0CLjKkyZB01iBc/aLgTvrtdvqaAHX9ifM1TVEo6/HsLd6lq/LUHz7VsJrZljDpyCDihvwM6FbCSlOS8UyiaNuJRAWX1YY+8QSmoX8EzVIgRh+6ZX2RlY7FIKC+WgsGT9/OvRIQ1T2Tjv0oOXfnyp2rghgplKgOX4Yk2mVWQF6QMHakH8r6DPwVRkgwOP33YYz5ljGabYsFdPkC8xgqtl1tetKgV1kPwM1iikRpfoTnyS53JiEyu5h1HNo+khB3DFNCqZcdl0eG8gsAJ2tXXWPCVa+Pz9F9xbfZzfM/OQsk0Grb6Xsv5y4Md/AN/qaK/JDSpho2W9QtQ296ujy37u7ufg/gagH2amLfwuHcvgQXIgs6G+sB2/CEDbTzX3qf4NKad3zs='} 2023-07-06T20:16:24.046 DEBUG:teuthology.nuke:shortname: smithi153 2023-07-06T20:16:24.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T20:16:24.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328147', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:36.817172', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbolPH+s+kUvv9GKmqZ24iklGKuOHmgaxXHyTohmTDlgsm5f43LwCq1OPDjKu6ZcN9ZRN24+VCCBQGETHv2hQFBvv2ut98zW19sVLuzpj1t8CFfr+1i2ZrRuU/NdFNuntkay/Z47cfg6ad7qIVH/EV9YA0G6XDCfHVN58isouGsLxWP3EmLDZnwxyLgGMUYn9I11qtLHCTu5Ru5lPUcbYuFm7I1NLiGSJU85I/lIjLEURU1awEB11ANyeBFIeZcr/+UawngE76MtYinp5PfiBKD/RXagAVXNioeD2CHpqqmKU/SfXK+ufDMby/PjnWL+lp+1wZ9b1Oqv0IXYJ5YTIk/kDdV9OPtK0CAGP3Dnn3Wf6optYpEsqpHsMbPaEF1xotK6ZVslPxNHpcPKYaRpWOiTOydPnvgIbttvBjKxzG21P6LSt/5YtikruWUfCUAzz8xbMNb7nWsACYRbkJAcmqkShJvYMuIMqI+PgV9tsN0fjSMR65yeYadmWpb+1LVTs='} 2023-07-06T20:16:24.107 INFO:teuthology.orchestra.console:Power off smithi027 2023-07-06T20:16:24.108 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T20:16:24.131 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:24.150 INFO:teuthology.orchestra.console:Power off smithi153 2023-07-06T20:16:24.151 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T20:16:24.174 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:28.164 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:28.198 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:32.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:32.311 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:32.524 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2023-07-06T20:16:36.396 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:16:36.608 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2023-07-06T20:16:36.714 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd client.1-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} duration: 1327.8946387767792 failure_reason: 'Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=fc6b66ab070a493faed86997001cf288 status: fail success: false 2023-07-06T20:16:36.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T20:16:36.809 INFO:teuthology.run:FAIL